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SUBJECT: 


Results  of  the  Focus  Group/Survey  of  the  Automated 
Patent  System 


I  want  to  thank  the  150  employees  who  participated  in  the  focus  group  and  the  over  700 
employees  who  responded  to  the  survey  evaluating  the  Automated  Patent  System  (APS). 
The  involvement  was  widespread  throughout  the  examining  disciplines  and  showed  a 
strong  endorsement  for  the  reliability  of  APS  and  supporting  networks.  Your  responses 
also  indicated  that  some  of  the  areas  that  we  need  to  improve  to  better  serve  the 
examining  corps  include:  refresher  training;  operability  and  maintenance  of  the  printers; 
and  connect  times  at  the  image  stations.  Efforts  are  already  under  way  to  correct  these 
problems  and  to  take  advantage  of  the  flexibility  of  the  APS  system  to  meet  new 
expectations  and  requirements  as  developing  technologies  emerge.  I  will  keep  you 
posted  on  our  progress  as  we  work  to  strengthen  APS  in  the  months  ahead. 

Automated  tools  enhance  the  productivity  and  quality  of  the  work  product  while 
providing  exciting  new  methods  to  perform  their  assigned  tasks.  I  greatly  appreciate  the 
time  and  efforts  that  you  contributed  to  help  improve  the  system  for  everyone,  and  look 
forward  to  continuing  this  dialog  as  we  prepare  the  Patent  and  Trademark  Office  to  meet 
the  challenges  of  the  next  century  and  beyond. 


VeSUtit  $-Uwh-~ 


Bruce  A.  Lehman 


PREFACE 


Since  1983,  our  group  has  had  the  good  fortune  to  assess  the  impact  of 
technology  in  many  diverse  situations.   We  have  held  discussions  with  consumers 
reacting  to  electronic  products  that  are  easily  available  in  the  marketplace.   We  have 
spent  hours  hearing  about  the  opportunities  and  challenges  surrounding  the 
introduction  of  advanced  telecommunications  and  automation  tools  in  America's  most 
prestigious  and  successful  corporations. 

No  matter  the  setting,  we  have  always  relied  upon  what  seems  to  us  to  be  a 
rather  straightforward,  common-sense  approach  to  technology  assessment.   We  have 
been  reluctant  to  frame  our  evaluations  around  what  senior  managers,  systems 
engineers  or  vendors  might  believe  are  the  right  measurements  for  success.    Instead, 
we  have  been  inclined  to  frame  our  line  of  inquiry  around  what  the  primary  users  of 
any  technology  --  both  internal  and  external  users  --  say  about  whether  the  electronic 
tools  under  review  are  meeting  their  business  needs. 

Through  the  years,  our  methodological  strategy  has  been  re-enforced  by  one 
basic  and  over-arching  assumption:   technology  is  meant  to  serve  people  and  not  vice 
versa.   To  us,  this  means  that  evaluation  measurements  should  be  crafted  around  the 
real-world  experiences  of  those  who  must  actually  rely  on  the  technology.   These 
users  rarely  initiate  discussions  about  capacity,  throughput,  CPU  expansion  or 
architectural  configurations.    Because  real  people  want  to  talk  about  the  end  results  - 
the  effect  of  technology  on  their  lives--  we  focus  on  their  issues  first  and  foremost. 
Our  first  objective  is  to  understand  effect  as  revealed  to  us  by  those  who  use  the 
technology  as  part  of  their  daily  lives.   Only  later  can  we  begin  to  investigate  cause 
and  establish  priorities  about  things  most  in  need  of  change  in  order  to  better  utilize 
the  technology  under  review. 

In  conducting  this  review  of  the  U.S.  Patent  and  Trademark  Office's  (PTO) 
Automated  Patent  System  (APS),  we  worked  within  the  same  philosophical  paradigm. 
Senior  officials  at  PTO  or  PRC  (the  system  developer)  were  not  asked  --  and  did  not 
attempt  --  to  establish  the  criteria  around  which  this  evaluation  effort  would  be  formed. 
They  did  agree  that  the  illuminating  force  of  the  study  would  be  the  issues  raised  by 
the  front-line  internal  users  of  APS  --  the  twenty-one  hundred  patent  examiners  whose 
experiences  with  APS  are  clearly  a  telling  measurement  of  the  project's  success  or 
failure. 


Toward  that  end,  our  report  has  taken  into  account  the  views  of  nearly  900 
examiners  --  almost  half  of  the  entire  examining  community  --  gathered  through 
personal  interviews  and  analysis  of  their  survey  responses  and  written  comments. 
Once  their  concerns  came  into  sharper  focus,  we  reviewed  a  number  of  pertinent  PTO 
documents  about  the  performance  of  certain  aspects  of  the  APS  as  a  basis  for 
validating  or  raising  questions  about  some  of  the  conclusions  drawn  by  the  examiners. 
These  data  are  reported,  where  appropriate,  in  this  report. 

This  is  our  report  of  how  well  the  APS  system  is  meeting  the  business  needs  of 
its  primary  users  --  the  examining  corps.   This  is  their  story  and  it  has  been  our 
welcomed  challenge  to  report  it  with  as  much  clarity  as  our  resources  and  time  allow. 

Patricia  Mathis 
Mathis  &  Associates 
January,  1995 
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Chapter  1:     INTRODUCTION 


The  U.S.  Patent  and  Trademark  Office  (PTO)  is  one  of  the  leading  agencies 
engaged  in  managing  our  nation's  intellectual  resources.    Responsible  for  examining 
patent  applications  and  issuing  patents,  PTO  received  186,123  patent  applications  in 
1994  -  a  number  that  is  expected  to  grow  at  an  annual  rate  of  4  percent.    Over  one 
million  documents  are  added  to  PTO's  patent  files  each  year.    Managing  these 
documents  and  processing  the  ever  growing  stream  of  patent  applications  has 
become  one  of  PTO's  greatest  challenges. 

As  a  result  of  a  legislative  mandate1,  PTO  initiated  an  automation  program  in 
1983.   The  Automated  Patent  System  (APS)  --  designed  to  improve  internal  PTO 
processes,  improve  patent  quality  and  to  facilitate  public  access  to  patent  information 
--  is  comprised  of  five  major  automated  information  systems2: 

■        Text  Search  -  This  system  allows  examiners  and  public 

customers  to  search  the  full  text  of  over  1.8  million  U.S.  patents 
issued  since  1971  and  the  English  translations  of  some  3.0  million 
Japanese  Patent  Office  abstracts.    {This  research  tool  has  been 
introduced  to  the  examining  corps  in  incremental  phases 
beginning  in  1986.   An  enhancement  project  to  replace  the  current 
text  search  facility  -  including  hardware  and  software  --  is 
scheduled  for  deployment  in  the  period  FY1995-FY2000.) 

■         Classified  Search-Image  Retrieval  (CSIR)  -  This  system 

provides  image  searching  of  over  5.8  million  patents3  on  dual 
screen  workstations  to  four  of  the  sixteen  examining  groups  or 
approximately  23%  of  the  examining  corps.   {It  was  introduced  in 
incremental  phases  beginning  in  1988.   Desktop  deployment  to  all 
examiners  is  scheduled  to  begin  in  FY  1996  and  be  completed  by 
FY1999.) 


In  1980,  Congress  enacted  Public  Law  96-517  which  required  PTO  to  develop  a  plan  for  the 
automation  of  all  PTO  operations. 

PTO  is  in  the  process  of  enhancing,  modifying  or  replacing  many  of  the  subsystems  within  each  of 
these  larger  systems.   Please  see  the  schedule  of  PTO  enhancement,  replacement  and  modification 
activities  for  FY1995-FY2000  in  Appendix  B  for  further  details. 

3PTO  currently  adds  some  2,000  patents  to  CSIR  each  week. 


■  Classification  Data  Systems  (CDS)  -  This  system  makes  it 
possible  to  maintain  classification  data  for  all  patent  and 
application  documents4  within  a  unified  database.    (This  system 
was  first  introduced  in  1982  and  updated  in  1993.   An  incremental 
enhancement  project  to  improve  hardware  and  software  is 
scheduled  for  deployment  over  the  period  FY1995-FY1999.) 

■  Global  Patents   -  This  system  will  ultimately  make 
domestic  and  international  patent  information  available  to 
the  examiner  on  the  desktop.   (Expected  deployment: 
FY1995-FY2000.) 

■  Patent  Application  Management  (PAM)  -  When 
implemented5,  this  system  will  fully  automate  the  patent- 
application  process,  including  the  filing  of  a  patent 
application  and  managing  the  application  through  PTO. 
This  system  will  eventually  replace  the  current  Patent 
Application  Location  and  Monitoring  System  (PALM). 
(Expected  deployment:  FY1994-FY2000.) 

These  systems  are  the  primary  automated  tools  used  by  PTO  in  the  patent  process. 


4PTO  reclassified  almost  400,000  U.S.  patents  in  1993.    Reclassification  is  the  process  of 
continually  revising  the  U.S.  Patent  Classification  System  to  maintain  its  usefulness  as  a  relevant 
search  system.   The  U.S.  Patent  Classification  System  is  an  extensive  schedule  of  patents  broken 
down  by  type  of  technology  into  classes  and  subclasses.  The  reclassification  process  involves  the 
restructuring  or  creation  of  classes  and  subclasses  of  patents  to  reflect:  1)  the  evolution  of  the  art,  or 
technology,  involved;  2)  the  emergence  of  new  art;  or,  3)  the  need  to  subdivide  subclasses  that  have 
become  too  large  for  efficient  search  by  examiners  or  other  users. 

5ln  response  to  an  agreement  between  the  PTO  and  the  Japanese  Government  to  immediately 
develop  and  deploy  a  system  to  provide  electronic  access  to  patent  applications  eighteen  months  after 
they  have  been  on  file,  PTO  launched  the  Pre-Grant  Publication  (PGPub)  system  project  and  put 
development  of  PAM  on  hold.   In  FY1996,  PTO  will  reassess  the  PAM  project  based  on  an  evaluation 
of  the  PGPub  system.   PAM  will  be  developed  under  the  Systems  Maintenance  and  Development 
(SDM)  contract,  which  will  be  awarded  in  1996.   The  SDM  vendor  will  eventually  replace  all  PTO 
automated  systems  vendors  and  be  responsible  for  the  development  and  maintenance  of  all  such 
systems. 


Patent  application  process 

At  the  end  of  FY1994,  PTO  had  received  186,123  patent  applications  --  a 
number  which  is  projected  to  grow  at  an  annual  rate  of  four  percent.   The  PTO  has 
set  as  its  goal  the  examination  of  these  patent  applications  within  eighteen  months6 
("patent  pendency").   Once  an  application  is  received  in  PTO's  mail  room,  it  is  given  a 
serial  number  and  an  eight-digit  application  number.   Fees  submitted  with  the 
application  are  sent  directly  to  PTO's  Finance  office. 

Following  the  serialization  process,  an  application  is  sent  through  the  pre- 
examination  process  where  it  is  classified,  security  screened  and  reviewed  for 
completion.    PTO  employs  approximately  12  pre-examination  classifiers  who  analyze 
the  subject  matter  of  a  patent  application,  decide  if  it  is  a  chemical,  electrical, 
mechanical  or  design  patent,  and  issue  a  primary  (class)  and  a  secondary  (sub-class) 
classification.     These  classifications  are  primarily  based  on  the  U.S.  Patent 
Classification  System. 

Pre-examiners  screen  the  application  for  security  and  government  property 
rights  and  for  all  other  required  parts.   At  this  point,  the  application  is  entered  into 
PTO's  current  application  tracking  program,  the  Patent  Application  Location  and 
Monitoring  (PALM)  system.   The  PALM  automated  system  generates  notices  which 
are  sent  to  the  applicant  throughout  the  patent  process,  such  as  the  file  receipt  that  is 
sent  out  once  the  application  is  reviewed  for  completion.   The  file  receipt  informs  the 
applicant  of  their  application  number,  the  official  filing  date,  the  application's  primary 
and  secondary  classification  numbers,  and  calls  the  applicant's  attention  to  any  parts 
of  the  application  that  may  be  missing. 

In  the  final  step  of  the  pre-examination  process,  the  application  is  microfilmed 
and  the  file  is  assigned  to  an  examining  group,  art  unit7,  and  an  individual  examiner 
who  will  ultimately  make  the  final  patentability  determination.    PTO's  patent  examining 
corps  is  broken  down  into  sixteen  examining  groups,  each  of  which  is  described  by  a 
four  digit  number.   Examining  groups  are  managed  by  a  director  and  organized  by 
discipline.   Chemical  group  numbers  begin  with  #1,  electrical  with  #2,  and  mechanical 
with  #3,  as  illustrated  in  the  table  that  follows  on  the  next  page: 


6See  the  PTO's  Business  Reengineering  Team's  Patent  As-ls  Model:  Reader  Kit  for  a  thorough 
tracking  of  the  patent  application  process. 

7An  art  unit  refers  to  the  sub-group  of  a  larger  examining  group  and  will  be  described  in  greater 
detail  later  in  this  section. 


Chemical  Examining  Groups 

1100  General,  Metallurgical,  Inorganic,  Petroleum  and 

Electrical  Chemistry  and  Engineering 
1200  Organic  Chemistry  Drug,  Bio-Affecting  and  Body 

Treating  Composition 
1300  Specialized  Chemical  Industries  and  Chemical 

Engineering 
1500  High  Polymer  Chemistry,  Plastics,  Coating, 

Photography,  Stock  Materials  and  Compositions 
1800  Biotechnology 

Electrical  Examining  Groups 

2100  Industrial  Electronics,  Physics  and  Related  Elements 

2200/2900     Utility/Design8 

2300  Information  Processing,  Storage  and  Retrieval 

2400  Packages,  Cleaning,  Textiles  and  Geometrical 

Instruments 
2500  Electronic  and  Optical  Systems  and  Devices 

2600  Communications,  Measuring,  Testing  and 

Lamp/Discharge 

Mechanical  Examining  Groups 

3100  Handling  and  Transporting  Media 

3200  Material  Shipping,  Article  Manufacturing,  Tool 

3300  Surgery,  Animal  Husbandry,  Medical  Technology, 

Amusement  and  Exercise  Devices  and  Printing 
3400  Solar,  Heat,  Power  and  Fluid  Engineering  Devices 

3500  General  Construction,  Petroleum  and  Mining 

Engineering 


8Group  2200  and  Group  2900  are  both  under  the  management  of  one  director.   Group  2200  is 
comprised  of  utility  patent  examiners  in  the  mechanical,  electrical  and  chemical  arts.   All  Group  2200 
examiners  are  required  to  have  a  special  security  clearance  because  of  the  nature  of  the  patents  they 
examine.   Group  2900,  the  Designs  patent  group,  has  distinct  responsibilities.   Because  design 
examiners  only  look  at  design  and  never  consider  utility,  they  use  APS  rarely. 


Each  examining  group  is  broken  down  into  four  to  sixteen  smaller  subgroups, 
called  art  units.   Art  units  are  also  represented  by  a  four-digit  number.    For  example, 
art  unit  1 302  (Food  or  edible  material,  processes,  compositions  and  products)  is  one 
of  the  eight  subgroups  which  make  up  Group  1300  {Specialized  chemical  industries 
and  chemical  engineering).   A  Supervisory  Patent  Examiner  (SPE)  is  assigned  to  head 
each  art  unit.   There  are  currently  a  total  of  150  SPEs. 

The  patent  application  examination  process  begins  when  a  patent  examiner 
receives  an  application  to  determine  patentability.   Examiners  must  rely  upon  a  vast 
body  of  technical  literature  to  make  this  decision,  including: 

■  Internal  sources,  such  as  the  patent  search  files  as 
accessed  through  APS  Text  Search  and  Image  Search,  and 
PTO's  manual,  paper-based  search  system;  and, 

■  External  sources,  such  as  commercial,  technical  computer 
databases  and  locally  and  remotely  located  technical 
libraries. 

This  body  of  internal  and  external  technical  literature  includes  U.S.  and  foreign 
patents,  as  well  as  non-patent  literature,  such  as  journal  articles.   The  latter  has 
become  an  increasingly  important  research  tool  in  the  examination  process, 
particularly  for  examiners  in  art  areas  that  are  rapidly  growing,  such  as  computer 
systems  and  biotechnology. 

Examiners  search  the  technical  literature  for  relevant  prior  patents  or  related  art 
in  order  to  determine  whether  or  not  a  patent  can  be  issued.   The  ease  with  which  an 
examiner  can  locate  relevant  prior  art  is  influenced  by  the  various  classification 
schemes  presently  used  to  organize  U.S.  and  foreign  patents  and  non-patent 
literature.   These  include  the  U.S.  patent  classification  system;  key  word  classification 
schemes  for  computer  arts;  specialized  databases  containing  the  full  text  of  more  than 
one  million  U.S.  patents;  an  electronic  database  of  biotechnology  information 
organized  by  RNA/DNA  features;  the  Library  of  Congress  indexing  system  used  in 
PTO's  Scientific  and  Technical  Library  (STIC);  and,  the  numeric  files  of  foreign  patent 
and  other  prior  art.9 

Hard-copy  and  on-line  sources  are  also  provided  to  examiners  to  facilitate  their 
identification  of  appropriate  art  areas  to  search.   These  include  the  Index  to  the  U.S. 
Patent  Classification  System,  the  Manual  of  Classification,  and  the  Manual  of 
Classification  Definitions. 


9See  the  Business  Reengineering  Team's  Patents  As-ls  Activity  Model:  Reader  Kit,  section  A0. 


Depending  on  their  seniority  and  their  art  unit,  each  examiner  is  given  a 
specified  number  of  hours  within  which  they  must  review  and  close  an  application.    In 
general,  examiners  spend  approximately  one-third  of  the  time  they  have  been 
allocated  for  each  application  in  identifying  what  art  they  should  research  They  spend 
another  one-third  actually  researching  and  the  remaining  third  writing  up  their  decision 
on  the  application.10  The  following  section,  "How  Examiners  Use  Automated  Tools  in 
the  Search  Process,"  describes  the  paper-based  and  automated  tools  used  to  search 
the  available  technical  literature  during  the  examination  process. 

Once  a  patentability  decision  has  been  made  on  an  application,  the  examiner  is 
responsible  for  drafting  a  series  of  administrative  office  actions  which  include  their 
decision  on  the  application's  patentability  and  documentation  of  the  references 
reviewed  in  the  search  process. 

During  the  post-examination  process,  applicants  are  informed  of  the  decision 
made  by  PTO  on  their  application.   Patent  documents  are  prepared  for  publication  -- 
in  such  journals  as  the  Patent  Official  Gazette  --  and  for  dissemination,  including  entry 
into  the  Automated  Patent  System  or  the  manual,  paper-based  search  system  used  by 
examiners.     Hard  copies  of  the  patents  are  made  available  to  the  public  in  PTO's 
Public  Search  Room,  located  in  their  Crystal  City,  Virginia  complex,  or  at  the  78 
Patent  and  Trademark  Depository  Libraries  (PTDLs)  located  across  the  country. 

Role  of  automation  in  the  patent  process 

Following  the  enactment  of  Public  Law  96-517,  which  required  PTO  to  fully 
automate  the  operations  of  the  agency,  an  Automation  Master  Plan  was  submitted  by 
PTO  in  1982  and  it  was  subsequently  approved  by  Congress. 


10lt  should  be  noted  that  this  time  allocation  figure  is  a  generalization.  The  amount  of  actual  time 
spent  on  each  of  these  tasks  varies  from  art  unit  to  art  unit,  from  application  to  application,  and  from 
examiner  to  examiner. 


PTO's  initial  Automation  Master  Plan  outlined  the  agency's  overall  automation 
strategy  for  the  period  1982-1990.   According  to  this  document: 

The  objectives  of  the  automation  program  are  designed  to  support  more 
general  objectives  established  for  the  PTO  itself.   These  are  to  increase  the 
quality  of  the  patents  and  trademarks  themselves  and  to  decrease  the 
pendency  of  patent  and  trademark  applications.    Patent  pendency  is  to  be 
reduced  to  eighteen  months.   The  automation  program  can  help  achieve  the 
quality  objective  by  eliminating  or  minimizing  the  inconsistencies  and  errors  in 
the  PTO's  files,  by  improving  the  procedures  used  to  correlate  incoming 
documents  with  pending  applications,  and  by  providing  examiners  with  access 
to  databases  that  are  more  complete  and  current  than  existing  databases.11 

Five  automated  subsystems  were  outlined  in  the  original  plan,  including:    a 
subsystem  to  capture  text  and  image;  a  subsystem  that  would  process  the  text  and 
image  databases;  a  subsystem  of  electronic  workstations  and  services;  a  subsystem 
that  would  provide  management  information  services;  and  a  subsystem  of  network  and 
process  control.12 

Although  the  overall  objectives  outlined  in  the  1982  Automation  Master  Plan 
have  continued  to  guide  PTO  in  the  development  and  implementation  of  its  automated 
systems,  the  actual  system  component  goals  have  changed  over  the  course  of  time. 
In  response  to  technological  changes  and  agency  requirements,  the  plan  was  revised 
several  times,  the  last  of  which  was  in  1987.    More  recently,  PTO  outlined  its 
automation  objectives  in  a  document  called  the  Strategic  Information  Technology  Plan: 
FY1995-FY2000™ 


11  U.S.  Patent  and  Trademark  Office.   Automation  Master  Plan:   Developed  in  Response  to  Section 
9  ofP.L.  96-517.    U.S.  Department  of  Commerce,  October,  1982,  page  1-4. 

12U.S.  Patent  and  Trademark  Office.   Automation  Master  Plan:   Developed  in  Response  to  Section 
9  ofP.L.  96-517.    U.S.  Department  of  Commerce,  October,  1982,  page  1-6,  1-7. 

^Strategic  Information  Technology  Plan:  FY1995-FY2000.   U.S.  Patent  and  Trademark  Office, 
October  24,  1994.  The  major  project  goals  relating  to  APS  and  the  patent  process  are  outlined  in 
Appendix  B. 


From  the  outset,  PTO  planned  to  develop  its  automation  program  in  an 
evolutionary  fashion  and  to  work  closely  with  the  private  sector  in  system  development 
and  implementation.   While  the  MITRE  Corporation  played  an  instrumental  role  in  the 
development  of  PTO's  initial  Automation  Master  Plan,  the  contract  to  begin  developing 
APS  was  ultimately  awarded  to  PRC,  Inc.  in  1984.   APS  has  been  developed  and 
deployed  incrementally  over  the  past  eleven  years  by  PTO,  PRC  and  other  vendors  as 
changes  in  technology  have  occurred  and  as  the  patent  process  itself  has  been  re- 
engineered. 

Automation  plays  a  major  role  in  the  patent  process.   In  addition  to  the  desktop 
PCs  that  are  used  by  managers,  administrative  staff,  examiners  and  classifiers 
throughout  PTO  for  a  mix  of  administrative  and  some  text  search  activities,  there  are 
three  major  automated  information  systems  (AIS)  currently  in  place  --  Classification 
Data  Systems,  Text  Search  and  Image  Search.   These  three  AIS  comprise  the  core 
parts  of  the  Automated  Patent  System  as  outlined  over  the  years  in  the  Automation 
Master  Plan  and  in  the  more  recent  Strategic  Information  Technology  Plan. 

The  Classification  Data  System  (CDS),  first  introduced  in  1982  and  then 
upgraded  in  1993,  maintains  classification  data  for  all  patent  and  application 
documents.    It  supports  U.S.  patent  classification,  foreign  patent  classification,  issued 
patent  lists,  and  patent  file  maintenance.    In  1993  alone,  PTO  reclassified14  almost 
400,000  patents  through  CDS. 

As  mentioned  earlier,  examiners  rely  extensively  on  classification  schemes 
throughout  the  examination  process.  They  are  able  to  access  CDS  using  Messenger, 
a  software  product  licensed  by  the  Chemical  Abstract  Service  (CAS),  in  order  to  relate 
patents  with  class  and  subclass  data.  PTO's  reclassification  projects  through  the  CDS 
analyze  thousands  of  patents  in  a  class  or  subclass  and  order  them  in  a  manner  that 
will  enable  an  examiner  to  most  efficiently  make  patentability  decisions.  PTO  plans  to 
initiate  a  four-year,  incremental  improvement  program  for  CDS  in  1995.15 

The  Text  Search  system,  introduced  in  1986,  allows  all  patent  examiners  to 
search  and  retrieve  --  through  shared,  single  screen  "text  terminals,"  image 
workstations  or  at  their  desktop  PC  --  the  text  of  over  1.8  million  U.S.  patents  granted 
since  1971,  as  well  as  the  English  language  abstracts  of  some  3.0  million  Japanese 
patents.   The  Text  Search  software  system,  called  Messenger,  is  a  licensed  product 
provided  by  PRC's  subcontractor,  Chemical  Abstract  Service. 


14See  Footnote  4  for  a  definition  of  reclassification. 


15See  Appendix  B  for  more  details. 


The  ability  to  search  the  full  text  database  was  first  provided  to  the  Public 
Search  Room,  located  in  PTO's  office  complex  in  Crystal  City,  Virginia,  in  1989  and 
later  in  1991,  to  14  Patent  &  Trademark  Depository  Libraries.16 

The  Classified  Search  and  Image  Retrieval  system  (CSIR),  or  Image  Search, 
allows  examiners  to  search,  retrieve,  display  and  print  the  images  of  5.8  million  U.S. 
patents17  dating  back  to  1790  using  double-screen  image  workstations.    Examiners 
can  also  search  the  text  of  patents  dating  back  to  1971  and  access  external 
commercial  databases  through  the  CSIR  terminals  or  "image  (or  cluster)  workstations." 
The  CSIR  system  is  comprised  of  the  Image  Search  database  and  CSIR  software. 

Image  Search  capability  is  currently  only  available  to  four  of  PTO's  sixteen 
examining  groups.   Group  2200  (Utility  Group,  which  includes  mechanical,  chemical 
and  electrical  arts)  was  provided  with  image  workstations  in  1988;  Group  2100 
(Industrial  Electronics,  Physics  and  Related  Elements  Group)  in  1 989;  and  Groups 
1200  (Organic  Chemistry  Drug,  Bio- Affecting  and  Body  Treating  Composition  Group) 
and  2300  (Information  Processing,  Storage  and  Retrieval  Group)  in  1993.    Groups 
2500  (Electronic  and  Optical  Systems  and  Devices  Group)  and  2600 
(Communications,  Measuring,  Testing  and  Lamp/Discharge  Group)  are  scheduled  to 
have  Image  Search  capability  by  March  13,  1995. 

Although  Image  Search  is  currently  not  available  through  desktop  PCs,  PTO  is 
scheduled  to  begin  a  four-year  desktop  deployment  project  in  FY1996.    By  the  end  of 
1999,  all  examiners  -  and  classifiers  --  should  have  image  search  capability  from  their 
desktop.    Image  Search  capability  has  existed  in  the  Public  Search  Room  since  1993. 

Public  users  can  place  orders  for  patent  documents  by  accessing  PTO's  Patent 
and  Trademark  Copy  Sales  (PTCS),  a  stand  alone  order  entry  and  production  system 
that  is  linked  to  APS's  Image  Search  system.   The  order  information  processed 
through  PTCS  is  passed  to  APS,  which  retrieves  the  images  of  the  ordered 
documents  from  CSIR's  high  resolution  database. 


16 

Fourteen  of  the  seventy-eight  Patent  and  Trademark  Depository  Libraries  (PTDL)  were  given  free 
access  to  APS  Text  for  a  three  year  period,  which  ended  in  December,  1994.   As  of  January,  1995, 
subscriptions  to  APS  Text  became  available  to  the  entire  PTDL  community  on  a  fee  basis.   The  PTO 
expects  that  PTDLs  across  the  country  will  be  making  decisions  about  subscription  to  this  service  over 
the  next  year. 


17 


PTO  adds  some  2,000  patent  images  to  CSIR  each  week. 


The  two  not-yet-operational  systems,  which  make  up  the  Automated  Patent 
System  as  outlined  in  the  Automation  Master  Plan  and  in  the  current  Strategic 
Information  Technology  Plan:   FY1995-FY2000,  are  the  Patent  Application 
Management  System  (PAM)  and  the  Global  Patents  system. 

The  Patent  Application  Management  system  (PAM)  will  replace  paper-based 
manual  procedures  for  filing  a  patent  application  with  electronic  procedures.    It  will 
simplify  and  expedite  patent  examiner  office  actions  and  provide  an  enhanced 
mechanism  for  tracking  applications  and  patent  actions  throughout  the  patent  process. 
PAM  will  also  replace  the  Patent  Application  Location  and  Monitoring  system  (PALM), 
the  current  workflow  tracking  and  status  reporting  system  for  patent  application 
processing. 

According  to  the  most  recent  Strategic  Information  Technology  Plan:   FY1995- 
FY2000,  PTO  recently  decided  to  redirect  the  PAM  effort  toward  the  development  and 
implementation  of  a  Pre-Grant  Publication  (PGPub)  system  by  January  1,  1996.   The 
PGPub  system  is  being  developed  in  response  to  an  agreement  of  the  administration 
with  the  Government  of  Japan18  requiring  PTO  to  provide  electronic  access  to  all 
patent  applications  eighteen  months  after  they  have  been  filed.   As  the  PGPub  system 
will  constitute  the  initial  implementation  of  capabilities  associated  with  PAM,  PTO  will 
use  an  evaluation  of  it  in  FY1996  to  determine  whether  and  how  to  proceed  with  the 
PAM  project.    Development  and  deployment  of  PAM  is  expected  to  be  completed  by 
FY2000  --  if  resources  are  made  available  --  under  the  Systems  Development  and 
Maintenance  (SDM)  contract.19 


18 

The  PGPub  system  is  being  developed  in  response  to  an  agreement  between  the  PTO  and  the 
Japanese  Patent  Office  (JPO).   Under  the  agreement,  PTO  will:  1)  "publish  pending  patent  applications 
18  months  after  they  have  been  filed,  beginning  with  applications  filed  after  January  1,  1996";  and,  2) 
"expand  reexamination  proceedings  to  allow  greater  participation  by  third  parties".   By  January  1,  1996, 
the  JPO  will  "end  the  practice  of  allowing  third  parties  to  oppose  a  competitor's  patent  before  it  is 
granted"  and  "put  in  place  ...an  accelerated  patent  examination  procedure  that  will  enable  applicants  to 
obtain  disposition  of  their  patent  applications  within  36  months."   Under  the  agreement  the  JPO  will  also 
"end  by  July,  1995  its  practice  of  awarding  dependent  patent  compulsory  licenses."  See  the  cover  story 
of  the  September,  1994  issue  of  PTO  Pulse  for  more  details. 

19PTO  is  planning  to  replace  the  current  APS  contractor  and  several  other  related  contractors  with 
a  Systems  Development  and  Maintenance  (SDM)  contractor  by  the  end  of  FY1996.   The  SDM 
contractor  will  be  responsible  for  the  planning,  analysis,  design,  development,  enhancement  and 
maintenance  of  all  of  PTO's  automated  information  systems  and  supporting  technical  infrastructure.  An 
RFP  for  the  SDM  contract  is  expected  to  go  out  by  the  end  of  1994. 
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The  development  and  deployment  of  the  Global  Patents  system  is  expected  to 
begin  in  FY1995  and  end  in  FY2000.   The  Global  Patents  project  will  develop  an 
integrated  system  that  will  allow  examiners  to  access  European  and  Japanese  patents 
from  their  desktop.   The  project  will  be  implemented  in  an  incremental  fashion. 
Initially,  examiners  will  be  able  to  search  and  retrieve  an  English  language  abstract  of 
the  Japanese  or  European  abstract,  as  well  as  an  image  of  the  main  drawing.    Later, 
the  full  images  of  the  first  few  pages  of  the  patent  will  be  available  at  the  desktop. 
Examiners  will  also  have  available  to  them  a  full  array  of  foreign  patent  search  tools, 
including  technology  classifications,  indices,  thesauri  and  concordances.   PTO  also 
plans  to  make  this  system  available  to  the  public  through  the  Patent  and  Trademark 
Depository  Libraries  (PTDLs). 

Other  non-APS  automated  tools  regularly  used  by  examiners  in  the  patent 
process  are  the  external  electronic  search  databases  which  contain  non-patent 
literature  (NPL)  and  foreign  patent  abstracts.   While  APS  does  not  have  an  internal, 
automated  NPL  search  and  retrieval  capability,  it  is  linked  electronically  to  external, 
commercial  databases.   PTO  is  considering  launching  an  NPL  project,  which  would 
begin  in  FY199920. 

How  examiners  use  automated  tools  in  the  search  process 

Patent  examiners  --  PTO's  front-end  APS  users  --  conduct  their  patent  searches 
using  six  major  channels  or  sources  of  information:21 

Major  Channels  of  Information 

The  shoes 

Image  (or  cluster)  workstations 

Text  terminals 

Desktop  PCs 

Commercial  databases 

Walk-up  printers 


20 

See  Appendix  B  for  more  information. 

21  Four  channels  allow  access  to  APS  and  five  channels  depend  in  some  measure  on  automated 
systems. 
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The  primary,  non-automated  source  is  an  extensive  system  of  wooden  and 
metal  shoebox-sized  drawers  --  "the  shoes"  --  which  contain  copies  of  patents  dating 
back  to  the  time  patents  were  originally  granted  by  the  U.S.  Government  in  1790.   The 
shoes  also  contain  a  limited  collection  of  foreign  patents,  as  well  as  a  somewhat 
random  collection  of  related  non-patent  literature  (NPL),  such  as  journal  articles.   The 
shoes,  which  require  manual  searching  of  paper,  are  located  throughout  PTO's  various 
offices  in  Crystal  City. 

The  automated,  APS-accessible  sources  include  the  CSIR  "image  (or  cluster) 
workstations,"  which  allow  approximately  23%  of  the  examining  corps  to  access  both 
text  and  image  from  one  workstation;  the  shared-use,  single  screen  "text  terminals," 
which  allow  examiners  to  access  APS'  Text  Search  system  using  Messenger  software; 
desktop  PCs,  which  currently  allow  some  91%  of  the  examining  corps  to  access  Text 
Search  through  direct  dataline  connection  via  the  PTO  Network  (PTOnet)  or  through  a 
modem;  and  the  walk-up  printers,  which  allow  examiners  to  print  out  a  copy  of  text 
or  image  after  they  have  completed  a  search  on  a  cluster  workstation,  a  text  terminal, 
or  their  desktop  PC. 

The  final  automated  source,  available  to  examiners  in  varying  degrees,  are 
commercial  databases,  which  provide  access  to  non-patent  literature  and  foreign 
patents.    Examining  groups,  as  well  as  individual  examiners  within  a  group,  have 
access  to  different  commercial  databases,  based  upon  the  types  of  information  they 
need  to  search  on  a  regular  basis.   An  examiner  must  have  an  account  number  to 
access  any  given  commercial  database,  whether  from  the  cluster  workstation,  text 
terminal,  desktop  PC,  or  a  database  located  in  PTO's  Scientific  and  Technical 
Information  Center  (STIC). 

The  extent  to  which  individual  examiners  rely  on  APS  as  a  primary  research 
tool  depends  on  a  number  of  factors:   the  nature  of  their  art  area;  individual  search 
techniques;  an  examiner's  experience  in  researching  the  shoes;  the  availability  of 
documentation  in  the  shoes  at  any  given  time;  and  an  examiner's  computer  comfort 
level  and  knowledge  of  APS'  features  and  functions. 

Why  APS  was  launched  as  a  project 

PTO's  Automation  Master  Plan  outlined  several  objectives  that  would  be 
achieved  by  the  development  and  deployment  of  the  Automated  Patent  System.   The 
major  objectives  were  to  improve  patent  quality,  minimize  patent  litigation,  simplify  and 
improve  document  management,  and  to  facilitate  information  exchange  with  foreign 
patent  systems. 
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The  primary  focus  of  APS  has  been  on  improving  patent  quality.22  While  APS 
directly  supports  the  patent  examiners  in  conducting  more  accurate  and  thorough 
searches,  the  principal  beneficiaries  of  improvements  in  quality  are  the  public  and 
nation  as  a  whole,  through  a  strengthened  intellectual  property  system.    Increased 
productivity  and  trade  result  from  new,  high  quality  patents,  delivered  in  a  timely 
manner  to  wider  audiences. 

Another  overarching  objective  of  the  automation  program  is  to  reduce  costly 
litigation  involving  patent  rights,  as  well  as  keep  time  consuming  patent  litigation  off  of 
crowded  Federal  court  calendars.   The  complete  file  integrity  which  APS  was  designed 
to  provide,  as  well  as  simultaneous  access  to  all  patents,  permits  the  patent 
examiners  to  conduct  their  searches  in  a  manner  that  greatly  improves  the  quality  of 
issued  patents.    Under  the  manual  system,  an  estimated  5-7%  of  the  documentation 
was  missing  at  any  given  time,  due  to  misfiling  or  use  during  the  examination  process. 
Examiners  also  report  that  as  much  as  30%  of  a  file  can  be  "missing"  at  any  given 
time  and  the  overall  file  integrity  tends  to  vary  between  examining  groups. 

Automation  also  addresses  the  problem  of  document  management.    Documents 
that  once  filled  the  equivalent  of  two  football  fields  will  now  fit  in  one  room.   This  is 
especially  important  to  the  Patent  Copy  Sales  division,  where  the  warehousing  of 
documents  was  becoming  a  major  issue.   Not  only  is  the  quality  of  the  patents  they 
are  able  to  produce  superior  under  APS,  they  can  also  respond  to  client  requests 
electronically,  providing  a  faster  and  more  efficient  public  service. 

Conversion  of  all  documents  to  digital  form  has  made  it  faster  and  easier  to 
exchange  information  with  foreign  patent  systems  when  that  capability  is  deployed. 
According  to  a  PTO  spokesperson,  PTO  and  the  Japanese  and  European  patent 
offices  have  already  signed  protocols  and  agreed  on  standards  for  information 
exchange.   Stimulation  of  international  trade  and  related  business  development  is  an 
anticipated  outcome. 


22While  many  criterion  could  be  selected  as  the  basis  for  defining  "patent  quality,"  it  can  be 
measured  by  the  number  of  cases  that  go  before  the  Board  of  Patent  Appeals  and  Interferences 
(BOPAI).  The  ability  to  review  and  take  into  account  all  previously  awarded  patents  is  a  bedrock 
requirement  for  patent  quality.  The  ever-expanding  volume  of  patents  and  the  requirements  for  100% 
file  integrity  would  make  a  modern  standard  for  quality  virtually  impossible  without  automated  tools. 
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How  APS  has  evolved 

When  PTO  awarded  the  contract  to  build  the  Automated  Patent  System  to  PRC 
in  April,  1984  it  was  one  of  the  largest  and  most  technically  challenging  automation 
programs  in  the  federal  government.   The  first  production  version  of  the  system  was 
built  on  leading  edge  technology.    In  fact,  the  CRT  resolution,  dual-screen 
configuration  and  performance  requirements  for  the  examiner  workstation  were  not 
available  commercially,  so  the  initial  workstations  had  to  be  specifically  developed  for 
the  APS.   For  the  other  components  --  such  as  the  mainframe,  file  servers,  support 
processors,  image  storage  devices  and  communications  network  components  --  the 
latest  in  available  technology  was  procured.   An  open  systems  approach  was  used  so 
that  the  system  components  could  expand  to  meet  future  requirements  and  migrate 
with  technological  advances. 

The  first  production  delivery  for  the  APS  was  the  text  search  capability  for  word 
searching  all  the  U.S.  patents  from  1975  to  current  --  about  one  million  patents.   This 
initial  capability  was  provided  along  with  training,  first  to  the  examiners  in  Group  2200 
in  1986  and  by  1988  to  the  entire  examining  corps  (over  1,200  examiners  at  that 
time).   A  second  text  database  containing  over  600,000  English  abstracts  of  Japanese 
patents  was  made  available  in  1987.   Today,  that  database  has  over  3.0  million 
abstracts. 

The  APS  began  its  first  production  run  with  image  search  and  retrieval  on 
December  14,  1987,  with  fifty  patent  examiners  from  Group  2200.   As  one  of  PTO's 
fifteen  patent  examining  groups  in  existence  at  that  time,  Group  2200  was  selected  for 
first  implementation  because  of  the  wide  range  of  patent  applications  it  processes. 
These  examiners  could  search  and  display  full  patent  images  from  163,000  patent 
documents  loaded  to  twelve  priority  image  storage  devices  connected  via  high  speed 
Xerox  Network  System  (XNS)  protocol  based  communications,  the  performance  leader 
at  that  time. 
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After  the  initial  capabilities  were  delivered,  APS  system  engineers  began  work 
on  the  requirements  to  provide  the  capabilities  to  the  other  examining  groups,  to 
increase  the  volume  of  data  available,  to  expand  functional  capabilities,  and  to  provide 
access  to  the  public.   To  accomplish  this  expansion,  many  additional  system 
components  and  technological  upgrades  were  incorporated  into  the  APS. 

In  1988,  the  text  search  capability  was  piloted  in  the  public  search  room  and  by 
1989  was  placed  in  production  for  public  user  access  to  the  PTO  text  databases.    In 
1991,  an  equivalent  capability  was  given  to  non-local  public  users  by  an  APS  pilot 
project  that  provided  text  terminals,  communications,  and  text  search  access  to 
fourteen  Patent  and  Trademark  Depository  Library  (PTDL)  sites23  strategically  located 
across  the  United  States. 

In  1989,  PTO  began  a  major  effort  to  expand  the  system  to  provide  access  to 
images  of  all  U.S.  Patents  back  to  1790,  a  total  of  5.7  million  patent  documents  and  to 
expand  the  functionality  and  size  of  the  U.S.  Patent  text  database.   This  required  the 
purchase  of  vast  amounts  of  additional  storage,  scanning  and  loading  of  over  5  million 
patent  image  documents,  replacement  of  the  existing  mainframe  and  the  upgrade  of 
user  workstations  and  network  communications. 

By  the  end  of  1990,  a  great  deal  had  been  accomplished.   The  Group  2100  and 
2300  examiners  were  brought  on-line  with  over  400,000  additional  patents  placed  in 
image  storage.   Additionally,  the  U.S.  Patent  text  database  was  reloaded  and 
expanded  to  contain  patents  from  1971  to  current,  over  1.8  million  patents.    Now,  both 
the  image  and  text  databases  grow  at  the  rate  of  125,000  patents  per  year. 

In  1992,  the  backfile  load  of  all  5.7  million  U.S.  patents  was  completed  for  the 
primary  storage  on  high  density  optical  storage  devices  (called  juke  boxes).   The 
requirement  to  support  patent  copy  sales  from  this  database  --  in  conjunction  with  the 
Patent  and  Trademark  Copy  Sales  (PTCS)  system  --  was  accommodated  in  less  than 
a  year.   This  extension  to  the  APS  uses  high  speed  printers  to  extract  and  print  paper 
copies  of  approximately  3,000  patent  documents  (50,000  pages)  a  day  for  requesting 
public  users  and  organizations. 


23See  footnote  16  for  more  information. 
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In  order  to  meet  system  communication  requirements,  PTO  embarked  on  two 
initiatives  to  upgrade  its  communication  and  networking  capabilities.  They  were  the 
migration  of  the  APS  to  the  Department  of  Defense  Internet  Protocol  Suite  (TCP/IP) 
and  the  implementation  of  PTO-wide  network,  called  PTOnet24.   The  first  TCP/IP 
components  were  installed  in  production  in  1992  and  the  last  XNS  component 
removed  in  1994.    PTOnet  is  designed  to  provide  universal  connectivity  from  the  user 
desktop  to  the  APS  computing  facilities  as  well  as  to  vital  office  automation  services 
such  as  electronic  mail,  printing  and  file  server  sharing.25 

APS's  evolutionary  course  required  that  basic  equipment  be  periodically 
upgraded  to  meet  user  requirements.   Workstations  were  re-engineered  with  better 
monitors  and  faster  processors  at  more  cost  effective  prices.    In  1993,  Group  1200 
and  the  Public  Search  Room  were  supplied  with  re-engineered  workstations  for  image 
search  and  retrieval  capability.   APS  system  engineers  are  now  performing  final  tests 
on  a  Shared  Use  Systems  (SUS)  workstation  which  costs  much  less  than  one  third  of 
the  original  workstation  and  uses  the  latest  Intel  technology.    It  will  also  provide  the 
basis  for  delivering  desktop  access  to  all  the  APS  capabilities. 

In  the  spring  of  1994,  APS  completed  the  load  of  all  patent  images  (5.8  million) 
to  priority  image  storage,  an  on-line  rapid  access  image  storage  for  examiner  retrieval 
and  browsing.   The  new  SUS  workstation  --  when  combined  with  the  expanded 
PTOnet  network  and  the  fully  loaded  image  database  -  will  position  PTO  for  full  APS 
deployment. 


24 

Data  regarding  PTOnet  performance  is  included  in  Chapter  5:    Performance  Indicators  and  in 
Appendix  D:   PTO  Exhibits. 

25PTO's  desktop  PCs  are  connected  to  PTOnet  through  file  servers,  dedicated  computers  that 
control  all  programs  and  peripherals  in  the  network  and  allow  the  PCs  to  access  the  network.   Through 
the  addition  of  file  servers  to  the  system,  more  desktop  PCs  can  access  the  network. 
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Why  this  study  was  commissioned 

PTO  initiated  its  automation  program  eleven  years  ago.   Of  the  $1  billion  initially 
dedicated  to  PTO's  automation  program,  some  $481  million  has  been  spent  to  date  on 
the  development,  enhancement  and  operation  of  parts  of  APS.   While  the  system  is 
not  yet  fully  operational,  changes  in  the  way  search  is  being  conducted  by  patent 
examiners  and  in  the  state  of  information  systems  technology  have  led  PTO  to 
reassess  how  the  remaining  resources  should  be  directed  over  the  next  seven  years. 
In  the  coming  months  PTO's  management  must  review  options  and  make  decisions 
about  whether  and  when  the  agency  should: 

■  Add  foreign  patents  and  non-patent  literature  to  the  CSIR 
database; 

■  Develop  and  deploy  PAM; 

■  Integrate  PAM  and  CDS  with  Text  Search  and  CSIR 

■  Automate  and  continue  to  improve  the  U.S.  Patent 
Classification  System; 

■  Deploy  desktop  workstations  to  all  patent  examiners,  patent 
classifiers  and  other  users; 

■  Develop  the  Examination  Toolkit  which  will  include  software 
to  enhance  and  facilitate  access  to  patent  literature,  non- 
patent literature,  classification  data,  help  systems  and  office 
action  creation  tools; 

■  Replace  the  current  Text  Search  system; 

■  Make  global  patent  information  available  via  desktop  PCs 
and  workstations 

PTO's  approach  to  developing  APS  has  also  come  under  external  review  in 
recent  years.    In  1988,  an  Industry  Review  Panel  (IRP)  --  comprised  of  representatives 
from  private  industry  with  experience  in  developing  and  implementing  large  scale 
automated  systems  -  was  convened  by  the  U.S.  Department  of  Commerce  to  review 
the  APS  program.  PTO  undertook  a  number  of  changes  in  response  to  the  IRP  report 
and  in  a  follow-on  review  in  1990,  the  IRP  stated  that  although  "...PTO  has  made 
dramatic  improvements. ..several  essential  improvements  need  to  be  made."26 


260ne  1988  recommendation  of  the  IRP  was  that  PTO  manage  APS  using  an  evolutionary 
approach. 
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Last  year,  the  Government  Accounting  Office  (GAO)  was  asked  by  members  of 
Congress  to  "determine  PTO's  ability  to  adequately  establish  benefit,  cost  and 
schedule  baselines  [for  APS],  and  monitor  progress  against  these  baselines."27   In  its 
September,  1993  report,  the  GAO  concludes  that  "PTO  does  not  have  all  the 
necessary  processes  in  place  to  permit  sound  management  of  APS. ..The  inability  to 
effectively  measure  progress  against  defined  goals  and  [to]  determine  causes  for 
deviation  [from  these  goals]  forces  PTO,  Commerce,  and  Congress  to  make  major 
investment  decisions  using  uncertain  system  benefits,  uncertain  cost  and  schedule 
baselines,  and  uncertain  performance  against  baselines."28 

PTO's  automation  program  was  also  cited  more  recently  in  Computer  Chaos: 
Billions  Wasted  Buying  Federal  Computer  Systems,  an  October,  1 994  report  by 
Senator  William  S.  Cohen,  the  Ranking  Member  of  the  Subcommittee  on  Oversight  of 
Government  Management,  Senate  Governmental  Affairs  Committee: 

Patent  information  is  a  strategic  resource  which  has  a  great  bearing  on 
our  international  competitiveness.   Accessing  this  information  quickly  and 
streamlining  the  application  process  could  have  the  effect  of  speeding 
U.S.  products  to  market.   Unfortunately,  [the  APS  project],  begun  in 
1983,  is  being  overtaken  by  advances  in  technology.   Oversight  has 
been  inadequate  and  the  evolutionary  management  approach  being  used 
has  left  costs  and  schedules  uncertain  and  Congress  with  little 
appreciation  of  what  it  is  buying  at  the  end  of  the  program.29 

Both  of  these  reports  recognize  the  difficulty  in  measuring  performance  when 
using  an  evolutionary  approach  to  computer  acquisition.   One  of  GAO's  three  major 
recommendations  for  PTO  is  that  a  process  be  established  to  define  the  expected 
benefits  of  its  automated  information  systems,  and  then  to  measure  actual 
performance  against  these  expected  benefits.   This  includes  identifying  expected 
benefits  to  both  internal  and  external  users  for  each  planned  APS  investment  before 
any  money  is  actually  invested. 


27U.S.  Government  Accounting  Office,  Report  to  Congressional  Requestors,  Patent  and  Trademark 
Office:   Key  Processes  for  Managing  Automated  Patent  System  Development  Are  Weak,  September, 
1993. 

28 

U.S.  Government  Accounting  Office,  Report  to  Congressional  Requestors,  Patent  and  Trademark 
Office:   Key  Processes  for  Managing  Automated  Patent  System  Development  Are  Weak,  September, 
1993. 

29 

Cohen,  Senator  William  S.    Computer  Chaos:   Billions  Wasted  Buying  Federal  Computer 
Systems.    Subcommittee  on  Oversight  of  Government  Management,  Senate  Governmental  Affairs 
Committee,  October  12,  1994,  page  15. 
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Such  observations  were  among  the  reasons  for  PTO's  decision  to  conduct  an 
in-depth  and  impartial  review  of  the  APS  system  among  those  where  its  impact  was 
felt  most  --  the  front-line  patent  examiners  who  daily  rely  on  the  technology  to  perform 
their  job  and  the  public  who  use  the  technology  during  some  phase  of  the  application 
and  patent  process. 

PTO's  major  external  customers  --  law  firms,  inventors,  patent  researchers, 
investors,  corporations  and  other  citizens  --  were  surveyed  recently  in  a  customer 
satisfaction  survey  conducted  by  the  U.S.  Census  Bureau.30  The  Census  Bureau 
mailed  the  PTO  customer  satisfaction  survey  in  November,  1994.    Data  from  this 
survey  is  expected  to  be  available  to  PTO's  Center  for  Quality  Services  in  February, 
1995,  and  that  organization  will  issue  a  report  of  findings  shortly  thereafter. 

Mathis  &  Associates,  a  Washington,  D.C. -based  technology  assessment  group, 
was  commissioned  to  provide  measurable  data  on  the  levels  of  internal  user 
expectations  and  satisfaction  with  APS.   The  primary  target  of  this  study  was  therefore 
the  2,028-member  Patent  Examining  Corps.31   The  portion  of  the  review  addressed  in 
this  report  is  largely  limited  to  Text  Search  and  CSIR,  the  two  components  of  APS  that 
are  regularly  used  by  the  examining  corps  in  the  process  of  making  patentability 
decisions  on  patent  applications.   Other  aspects  of  APS  will  be  reviewed  at  a  later 
time. 

Methodology 

The  study  was  conducted  in  two  phases.    During  Phase  I,  Mathis  &  Associates 
conducted  a  series  of  interviews  and  focus  group  sessions  and  developed  a  draft 
survey  instrument.    During  Phase  II,  the  survey  instrument  was  pre-tested,  revised  and 
fielded.   The  results  from  the  survey  were  analyzed  and  this  report  was  written  during 
Phase  II. 


30 

The  PTO  external  survey  is  one  in  a  series  of  customer  satisfaction  surveys  the  U.S.  Census 
Bureau  is  conducting  for  the  U.S.  Department  of  Commerce. 

31This  includes  1,878  patent  examiners  and  150  supervisory  patent  examiners. 
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Conduct  of  interviews  and  focus  groups 

Of  primary  concern  in  developing  the  study  was  the  desire  to  develop  a  survey 
instrument  that  would  determine  the  extent  to  which  the  information  needs  of  the  end 
users  --  PTO's  examining  corps  --  are  being  met  by  APS.    In  August,  1994,  Mathis  & 
Associates  began  its  research  by: 

■  reviewing  the  relevant  material  on  APS  produced  by  PTO 
and  PRC,  such  as  the  March  31,  1994  "Analysis  of 
Advantages  of  APS,"  produced  by  the  Office  of  Planning, 
Development  and  Evaluation  of  the  Search  and  Information 
Resources  Administration,  as  well  as  the  September,  1993 
GAO  report,  "Patent  and  Trademark  Office:    Key  Processes 
for  Managing  Automated  Systems  Are  Weak;"  and, 

■  consulting  with  information  resource  technology  experts. 

In  addition,  interviews  were  conducted  with  over  150  people,  including  PTO 
managers  from  the  Search  and  Information  Resources  Administration  (SIRA)32,  the 
Center  for  Quality  Services33  and  the  Future's  Group34.   Participants  in  these 
interviews  were  asked  to  comment  on  their  impressions  of  how  patent  examiners  use 
and  view  APS.    Many  of  these  commentators  had  previously  served  in  the  patent 
examining  corps.    Others  had  been  involved  in  various  studies  and  surveys  of  APS35, 
and  in  planning  for  future  system  developments  and  enhancements. 


32PTO's  Search  and  Information  Resources  Administration  is  responsible  for  overseeing  and 
administering  the  Office  of  Search  Systems,  the  Scientific  and  Technical  Information  Center  (STIC),  the 
Classification  Groups  and  the  Office  of  Classification  Support.   For  the  purposes  of  this  study,  Mathis  & 
Associates  was  primarily  interested  in  the  views  of  managers  in  the  Office  of  Search  Systems, 
particularly  the  Evaluation  division.   This  division  determines  user  satisfaction  with  current  systems, 
assesses  user  needs  to  aid  in  automation  systems  development,  and  assesses  the  efficiency  and 
capabilities  of  automated  systems  and  their  effects  upon  Patents  Cost  Center  operations,  work  loads 
and  work  patterns. 

PTO's  Center  for  Quality  Services  provides  guidance,  coordination  and  assistance  for  ensuring 
quality  services  to  internal  and  external  customers.   The  center's  activities  include  assistance  in 
understanding  customer  needs  and  expectations  so  that  quality  services  can  be  defined;  establishing 
service  standards;  measuring  customer  satisfaction;  benchmarking;  and,  determining  the  policy  and 
requirements  of  necessary  training  for  a  quality-driven  work  force. 

PTO's  Future's  Group  is  an  advisory  body  comprised  of  Patents  Cost  Center  managers.   The 
group  advises  the  Assistant  Commissioner  for  Patents  on  automation  issues  and  needs. 

35Search  and  Information  Resources  conducts  on-going  and  ad  hoc  studies  on  APS  usage, 
including  regular  interviews  with  examiners  with  access  to  Image  Search. 
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Immediately  following  these  interviews,  Mathis  &  Associates  conducted  a  series 
of  seventeen  focus  group  sessions  with  randomly  selected  patent  examiners. 
Representatives  from  all  sixteen  examining  groups  participated  in  this  process,  which 
was  designed  to  be  an  impartial  forum  for  learning  directly  from  examiners  how  APS  is 
used  in  the  examining  process.   At  the  beginning  of  each  session,  Mathis  & 
Associates  explained  the  nature  of  the  project  and  stressed  that  the  objective  of  the 
focus  group  meeting  was  to  determine  the  extent  to  which  APS  was  meeting  their 
information  needs.    Participants  were  asked  to  frame  their  discussion  around  three 
basic  lines  of  inquiry: 

■  What  about  APS  is  working  well? 

■  What  about  APS  needs  to  be  improved? 

■  What  about  APS  would  you  change  if  you  could? 

This  framework  stimulated  much  lively  discussion  and  information  sharing  which  was 
captured  in  a  preliminary  report  of  findings. 

Development  of  the  survey  instrument 

Mathis  &  Associates  used  the  information  culled  from  these  interviews  and 
focus  group  discussions  as  the  basis  for  drafting  a  survey  instrument.   The  draft 
survey  instrument,  designed  to  elicit  data  about  individual  examiners  and  examining 
group  specific  information  needs,  was  delivered,  along  with  the  preliminary  report  of 
focus  group  findings,  to  PTO's  senior  management  team  on  September  19,  1994. 

Mathis  &  Associates  reviewed  the  survey  instrument  with  appropriate  PTO 
managers  on  September  20,  and  pre-tested  it  with  three  groups  of  randomly  selected 
examiners  on  September  21-22,  1994.    Examiners  completed  the  survey  and  then 
participated  in  a  group  discussion  about  the  content  and  style  of  the  instrument. 
Senior  PTO  managers  and  representatives  from  the  employee  union36  also  provided 
comments  on  the  survey's  design  and  content. 


36Patent  Office  Professional  Association  (POPA) 
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Mathis  &  Associates  revised  the  survey  instrument  based  on  the  comments 
provided  to  them  by  PTO's  designated  commentators.   The  final  survey  product 
contained  60  questions  and  over  302  data  elements,  divided  into  the  following 
sections: 

Section  I:   Overall  Information  Resources  Assessment  -  Satisfaction 
with  and  importance  of  various  aspects  of  the  information  resources 
available  to  examiners  at  PTO 

Section  II:    Research  Tools  Assessment  -  Level  of  usage  of  the  various 
channels  used  to  access  APS  and  satisfaction  with  and  importance  of 
how  each  of  the  channels  are  performing 

Section  III:    Future  Needs  Assessment  -  Estimated  impact  on  job 
performance  if  certain  features  or  enhancements  were  introduced  in  the 
future 

Section  IV:    Demographics  -  Demographic  information  on  the 
professional  and  personal  characteristics  of  respondents 

Section  V:   Comments  -  Elaboration  on  any  of  the  responses  to  the 
questions  in  the  survey 

Selection  and  design  of  the  sample 

All  members  of  the  examining  corps,  including  the  150  Supervisory  Patent 
Examiners,  were  provided  with  an  opportunity  to  participate  in  the  study.   The 
Assistant  Commissioner  for  Patents,  Lawrence  J.  Goffney,  Jr.,  sent  a  letter  to  all 
examiners  on  October  13,  1994  encouraging  their  active  participation  in  the  survey 
process.37  The  process  was  conducted  in  two  phases:   a  controlled  survey 
administration  and  a  mail-in  survey. 


37See  Appendix  F  for  a  copy  of  this  letter. 
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Fielding  of  the  survey 

With  the  assistance  of  PTO,  a  random  sample  of  500  examiners  --  representing 
all  sixteen  examining  groups  --  was  identified  to  participate  in  a  general  survey 
administration  session  on  October  27,  1994.   The  random  selection  was  generated  by 
PTO's  Center  for  Quality  Services  staff  using  Excel  software  and  a  list  of  2,028  patent 
examiners  and  supervisory  patent  examiners.    Each  of  the  identified  500  examiners 
received  a  letter,  through  their  group  director,  on  October  18,  1994  which  invited  them 
to  participate  in  the  general  session.   They  were  asked  to  respond  to  the  invitation  by 
October  24,  1994.    On  that  same  day,  the  group  directors  provided  the  Center  for 
Quality  Services  with  a  list  of  all  those  who  indicated  that  they  would  attend  the 
session. 

Three  hundred  forty-four  (344)  examiners  participated  in  the  October  27th 
session.   The  general  session  was  held  at  the  Patent  Academy  during  two  time 
periods,  10:00  a.m.  and  2:00  p.m.    Examiners  were  given  approximately  one  hour  to 
complete  the  survey.    Surveys  were  collected  by  Mathis  &  Associates  and  transported 
directly  to  their  office  for  data  entry. 

On  Monday,  October  24,  1994  the  Center  for  Quality  Services  mailed  the  same 
survey  to  the  homes  of  those  1,528  remaining  examiners  who  had  not  been  randomly 
selected  to  participate  in  the  October  27th  general  session,  ensuring  that  all  examiners 
received  the  survey  document  within  the  same  forty-eight  (48)  hour  period.   A  letter 
from  PTO  Commissioner  Bruce  Lehman38,  which  encouraged  examiners  to  complete 
and  return  the  survey,  accompanied  the  survey.    Examiners  were  provided  with  a 
postage-paid  envelope  addressed  to  Mathis  &  Associates.   (November  28th  was 
established  as  the  cutoff  date  and  surveys  received  after  that  date  were  not  included 
in  the  analysis  of  the  results.) 

Preparation  of  the  data 

A  total  of  three  hundred  forty-four  (344)  usable  questionnaires  were  returned 
from  the  October  27,  1994  administration.   A  total  of  three  hundred  eighty  (380) 
usable  questionnaires  were  returned  from  the  mail-in  process  for  a  response  rate  of 
24.8%.   The  total  overall  response  rate  from  both  the  general  session  and  the  mail-in 
survey  was  35.7%. 

Responses  were  key  punched,  100%  verified  and  edited  by  Mathis  Survey 
Group  according  to  Mathis  &  Associates'  specifications.   A  verification  audit  of  the  final 
data  set  was  performed  by  an  independent  third  party,  Action  Data  Processing,  Inc. 


38 

See  Appendix  F  for  a  copy  of  the  letter  from  Commissioner  Lehman. 
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Analysis  of  the  data 

The  survey  design  and  preliminary  analysis  of  responses  provided  the  basis  of 
analysis  of  the  data.   The  statistical  analysis  in  this  study  was  restricted  to  cross- 
tabulations. 

Confidentiality  and  anonymity  of  participants 

The  confidentiality  and  anonymity  of  respondents  were  preserved  throughout 
the  conduct  of  the  study.    Names  of  participants  in  the  October  27,1994  session  were 
randomly  selected  by  computer  and  subsequently  destroyed. 

All  participants  were  requested  not  to  put  their  names  on  the  survey  and  no 
identifying  numbers  were  placed  on  the  survey  that  could  be  traced  to  an  individual 
participant. 

Scoring  of  the  statistical  responses 

Respondents  were  asked  to  rate  their  satisfaction  with  various  elements  using 
the  following  scale: 

1  =  Very  satisfied 

2  =  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 
N/A  =  No  answer 

Examiners  were  also  asked  to  indicate  how  important  they  believed  each 
element  was  to  their  ability  to  perform  their  job,  using  this  scale: 

• 

1  =  Critically  important 

2  =  Very  important 

3  =  Somewhat  important 

4  =  Not  too  important 

5  =  Not  at  all  important 

In  other  sections,  respondents  were  asked  to  indicate  frequency  of  use  or 
estimate  the  impact  of  feature  enhancements  or  changes  if  they  were  made  to  the 
system. 
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In  reporting  statistical  scores,  responses  were  grouped  into  two  clusters. 
Responses  in  the  1,  2  and  3  categories  were  aggregated  to  report  an  overall 
"acceptance"  score  --  reflecting  the  fact  that  the  remainder  of  respondents  expressed 
some  level  of  dissatisfaction  with  a  given  element.    In  writing  the  report,  general 
"acceptability"  is  also  sometimes  reported  as  general  satisfaction  -  again  because  the 
rating  excludes  all  dissatisfied  respondents.   Responses  in  the  4  and  5  categories 
were  aggregated  to  report  an  overall  dissatisfaction  score. 

Reliability  of  the  data 

A  total  of  seven  hundred  twenty-four  (724)  completed  surveys  were  returned 
prior  to  the  November  28,  1994  cut-off  date  from  a  total  patent  examiner  population  of 
2,028.   This  response  level  produced  a  ±2.9  error  rate  at  the  95%  confidence  level. 
Formula  applied  to  derive  these  calculations  are  explained  in  Appendix  E:    Statistical 
Analyses  Derived  from  Survey. 

Other  factors  in  interpreting  the  data 

In  reviewing  the  statistical  data  in  this  report,  there  are  two  issues  which  should 
be  kept  in  mind.    First,  when  adding  the  percentages  of  responses  to  any  given 
question,  it  is  possible  that  the  total  will  equal  just  below  or  just  above  100%. 
Statistical  software  packages  used  to  analyze  data  routinely  round  off  each  individual 
response  to  the  nearest  whole  number. 

Second,  when  reviewing  the  cross  tabulations  of  responses  against  the  three 
broad  examining  group  categories  --  chemical,  mechanical  and  electrical  --  it  is 
important  to  note  that  the  percentages  of  respondents  from  each  of  these  broad 
groups  only  reflects  the  number  of  examiners  from  each  group  that  actually  answered 
that  question  and  also  indicated  to  which  broad  group  they  felt  they  belonged.   There 
was  a  small  group  of  examiners  who  did  not  identify  themselves  as  a  chemical, 
electrical  or  mechanical  examiner.   Therefore,  while  their  responses  were  included  in 
the  all  examiner  category,  their  responses  could  not  be  included  in  the  cross 
tabulations. 
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Chapter  2:     OVERALL  ASSESSMENT 


Examiners  developed  their  own  list  of  twenty-one  information 
resource  management  baseline  priorities 

During  the  focus  group  and  interview  phase,  front-line  examiners  and  their 
managers  were  asked  to  indicate  what  measurements  were  appropriate  for 
determining  how  well  APS  is  working  within  the  examining  corps.   These  groups 
consistently  raised  a  core  list  of  information  resource  management  issues  --  such  as 
the  adequacy  of  APS-related  training  and  the  effectiveness  of  the  help  desk 
organization.   Their  ultimate  list  of  twenty-one  priorities  also  included  some 
environmental  issues  ~  like  lighting,  cleanliness  and  climate  control  in  the  shared  work 
areas  --  based  on  their  argument  that  examiners  could  not  be  expected  to  use  APS 
well  when  they  were  physically  uncomfortable  in  the  space  where  the  equipment  was 
located.   They  also  produced  a  list  of  core  items  that  addressed  the  bedrock 
measurements  they  believed  were  appropriate  for  evaluating  how  well  APS  was 
actually  working  for  them. 

The  list  of  twenty-one  baseline  items  was  transformed  into  a  line  of  inquiry  that 
constituted  the  first  section  of  the  survey  instrument:   overall  information  resources 
assessment.    Results  from  this  section  --  which  is  depicted  graphically  on  the  next 
page  --  reveal  the  rank  order  of  satisfaction  levels  reported  by  examiners  for  each  of 
the  twenty-one  items.    Levels  of  importance  are  also  indicated  below  each  satisfaction 
score. 

Overall  APS  reliability  scored  a  general  level  of  acceptability 

Given  the  inclination  of  most  groups  to  select  APS  as  the  surviving  research 
tool  --  if  they  could  keep  only  one  --  it  is  not  surprising  that  survey  respondents  gave 
their  third  highest  satisfaction  rating  to  "overall  reliability  of  APS  and  its  supporting 
networks"  (82%  satisfied).   This  ranking  was  third  only  to  two  relatively  minor 
administrative  and  environmental  issues:   the  hours  of  operation  of  APS  and  the  walk- 
up  printers  (84%)  and  lighting  in  the  shared  facility  workspace  (88%  satisfied). 
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Training  satisfaction  is  a  mixed  message 

The  most  unmistakable  message  to  emerge  from  the  interview  phase  was  that 
examiners  need  more  and  better  training.   While  commentators  spoke  positively  about 
the  quality  and  general  effectiveness  of  their  initial  APS  training  experience,  virtually 
no  one  had  received  refresher  training.   The  need  for  more  on-going  training 
became  obvious  in  each  discussion  when  it  became  clear  that  many  examiners  lacked 
both  computer  skills  and  APS  training.   Therefore,  it  was  not  surprising  that  examiners 
rated  their  initial  training  experiences  in  the  highest  cluster  of  ratings  --  and  they  gave 
their  worst,  rock-bottom  score  (51%)  to  "refresher  and  update  training." 

Other  APS-related  measures  were  rated  across  the  spectrum 

Examiners  reported  acceptance  levels  that  exceeded  70%  with  all  critical  APS 
performance  elements  -  except  for  their  notable  disappointment  with  the  extent  to 
which  APS  had  been  customized  to  meet  individual  examining  group  needs  (57% 
satisfied).   Three  out  of  four  examiners  find  the  accuracy  of  the  classification  system 
to  be  acceptable.    Seven  out  of  ten  gave  relatively  favorable  ratings  to  APS'  overall 
user  friendliness.   While  this  rating  may  appear  on  its  face  to  be  unimpressive,  it  is 
important  to  compare  this  rating  with  those  given  by  examiners  for  all  of  the  other 
systems  they  routinely  use  to  perform  their  searches.   As  the  graph  below  illustrates, 
APS  compares  very  well: 


APS  user-friendliness  compared  to 
commercial  databases 


45a.  User-fnendlmess  of  CAS/STN 

1q  Overall  user-friendliness  of  APS 

45b  User-friendliness  of  Derwent 

45c  User-friendliness  of  Dialog 

45g  User-fnendlmess  of  Other 

45d  User-friendliness  of  Lexis 

45e  User-fnendlmess  of  Westlaw 

45f  User-friendliness  of  Orbit 


■  satisfaction 
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Examiners'  disappointment  with  how  APS  has  been  customized  to  meet  their 
highly  individualized  art  group  requirements  was  a  subject  that  occupied  much 
discussion  time  during  the  interview  phase.   Consequently,  it  came  as  no  surprise  that 
almost  half  of  all  examiners  (43%)  are  dissatisfied  with  APS'  customization  features. 
A  more  detailed  discussion  of  this  matter  is  provided  in  Chapter  9:    Future  Needs 
Assessment 

Commentators  in  the  focus  groups  alluded  frequently  to  the  lack  of  availability 
of  terminals  and  printers  when  they  needed  them  --  either  because  not  enough  existed 
or  because  poor  maintenance  kept  them  out  of  commission  much  of  the  time.   Survey 
results  indicate  that  a  majority  of  examiners  hold  these  views.   Thirty  percent  (30%) 
expressed  some  level  of  dissatisfaction  with  the  availability  and  maintenance  of  the 
terminals1  and  over  two-thirds  (37%)  are  dissatisfied  with  the  availability  of  printers 
when  they  need  them.   A  more  comprehensive  discussion  about  availability  and 
maintenance  issues  is  presented  in  Chapter  5:   Performance  Indicators. 

Some  communication  and  management  issues  affect  user  attitudes 
and  effectiveness  of  APS 

Focus  group  commentators  sometimes  barraged  the  interviewers  with  examples 
of  Help  Desk  ineffectiveness.   They  claimed  that  the  Help  Desk  organization  was 
understaffed  and  frequently  not  sufficiently  trained  to  quickly  diagnose  and  help  solve 
the  APS-related  problems  examiners  reported.   Yet  the  data  indicates  that  over  four- 
fifths  (82%)  of  all  examiners  are  satisfied  with  the  effectiveness  of  the  Help  Desk. 
This  discrepancy  between  the  anecdotal  observations  and  statistical  data  suggest  that 
for  those  examiners  who  most  need  help  and  cannot  find  it,  any  inadequacy  in  Help 
Desk  performance  is  likely  to  loom  larger  than  life. 

The  PTO  newsletter  --  often  criticized  during  the  interview  phase  as  being 
uninformative  and  not  especially  useful  in  educating  examiners  about  APS-related 
events  --  actually  received  a  vote  of  confidence  in  the  survey  itself.    Eighty-one 
percent  (81%)  of  all  respondents  saw  it  as  an  effective  tool  for  keeping  examiners 
informed  about  system  changes,  new  features  and  how  to  use  them. 

Examiners  were  more  critical  of  the  documentation  and  reference  materials  they 
need  to  use  the  equipment,  systems  and  databases  to  full  advantage  and  focus  group 
commentators  made  numerous  references  to  their  disappointments  and  requirements 
in  this  area.   Seventy-two  percent  (72%)  of  all  survey  respondents  reported  they  were 
either  satisfied  or  neutral  on  the  subject. 


1The  deployment  of  48  new  terminals  -  which  occurred  as  the  survey  was  being  fielded  --  should 
address  some  of  these  concerns. 


30 


Acceptance  scores  clipped  below  the  seventy  percent  (70%)  line  in  all  other 
communications  measurements  included  in  the  introductory  assessment  section. 
Sixty-six  percent  (66%)  of  respondents  said  they  were  "satisfied"  or  "neutral"  with 
opportunities  they  had  to  swap  information  among  colleagues  about  search  methods 
and  techniques.   This  scoring  was  somewhat  better  than  expected  given  the 
inclination  of  focus  group  participants  to  often  get  sidetracked  in  spontaneous 
seminars  as  they  enlightened  one  another  about  clever  ways  to  execute  searches  and 
use  APS  more  efficiently. 

Examiners  also  gave  lower  scores  (64%  acceptance)  for  "reliable  information 
about  system  changes  and  enhancements  scheduled  for  implementation."   Examiners 
claimed  in  personal  interviews  that  they  rarely  understood  the  episodic  changes  in 
hardware  and  software  and  that,  many  times,  unidentified  staff  came  during  the  night 
and  reconfigured  equipment  and  software  without  notice  or  explanation.   They  said 
these  events  often  leave  them  confused  for  days  about  how  to  use  the  new  processes 
and  that  they  are  usually  left  to  their  own  devices  to  determine  what  to  do  or  how  to 
use  the  new  and  sometimes  baffling  tools. 

Interviewees  frequently  said  they  were  not  encouraged  to  make  constructive 
recommendations  for  improving  APS  and  related  systems.    Many  were  only  vaguely 
aware  of  the  user  group  representative  program2  and  the  role  of  SIR3  in  listening  to 
and  articulating  examiner  suggestions  up  the  management  chain.  The  purpose  of  the 
"bug  report"4  surfaced  in  several  interviews  and  examiners  said  they  had  stopped 
submitting  these  reports  because  no  action  was  ever  taken.    Not  surprisingly  then, 
ratings  for  satisfaction  with  both  "feedback  on  suggestions"  (56%)  and  "availability  of  a 
channel  for  submitting  suggestions"  (53%)  were  among  the  lowest  ratings  emerging 
from  the  overall  assessment  section.   Some  of  this  distrust  was  openly  attributed  to  a 
management  philosophy  that  prevailed  in  recent  years  and  is  also  referenced  in 
Chapter  7:   Communication,  Management  and  Support  Issues. 


A  user  group  representative  is  a  member  of  the  professional  staff  of  the  Patent  Cost  Center  --  part 
of  the  Search  and  Information  Resource  Administration  --  who  represents  an  active  liaison  for  the  other 
members  of  the  Patent  Examining  Corps  to  ensure  that  their  automation  needs  are  properly  represented 
and  fulfilled  by  the  APS  development  staff. 


The  Evaluation  division  in  the  Office  of  Search  Systems  of  PTO's  Search  and  Information 
Resources  Administration  is  responsible  for  determining  user  satisfaction  with  current  automated  search 
systems  and  assessing  user  needs  to  aid  in  automation  system  development. 

4A  "bug  report"  is  a  user  generated  report  of  technical  problems  or  system  performance  problems 
encountered  with  APS,  such  as  catastrophic  system  errors,  slow  log  on  time,  or  unclear  patent  images. 
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Other  environmental  and  management  issues  were  among  other 
baseline  concerns  but  seen  as  less  significant 

Some  focus  groups  participants  complained  at  length  about  environmental 
conditions  in  the  workstation  space  --  some  claiming  it  was  too  cold,  others  it  was  too 
dark  and  still  others  who  alleged  the  space  was  rarely  clean.   However,  the  data 
indicates  that  over  three-fourths  of  all  examiners  are  satisfied  with  this  aspect  of  their 
work  life  and  do  not  see  these  factors  as  impediments  for  APS's  success. 
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Chapter  3:     USAGE  OF  RESEARCH  TOOLS 

No  single  search  tool  completely  fills  the  information  needs  of  the 
examining  corps 

Focus  group  commentators  consistently  stressed  that  no  single  search  tool  could 
currently  completely  fill  their  information  needs.   Search  requirements  vary  from  art 
unit  to  art  unit.     Shoe  searches  or  image  searches  are  required  in  some  while  word 
searching  simply  is  not  possible  in  others.   The  physical  state  of  the  shoes  varies  from 
art  unit  to  art  unit.    File  integrity  is  very  poor  in  some  areas.    In  some  disciplines, 
patent  information  rarely  exists  in  paper  form;  in  others,  it  only  exists  in  paper  form. 
These  variations  and  requirements  are  compounded  by  the  fact  that  individual 
examiners  have  different  search  techniques,  computer  comfort  levels,  knowledge  of 
APS's  features  and  functions  and  experience  in  working  with  the  shoes.     Not 
surprisingly,  the  statistical  data  confirmed  that  most  examiners  must  use  a 
combination  of  search  tools  for  each  patent  application.1 

Examiners  still  use  the  shoes  on  a  regular  basis 

Despite  their  widespread  dependence  on  APS,  focus  group  commentators 
reported  two  primary  reasons  why  they  continue  to  use  the  shoes.    For  examiners 
working  in  certain  arts,  the  shoes  contain  information  not  available  through  APS  -- 
primarily  non-patent  literature  and  foreign  art.     Some  groups  --  such  as  Group  1800 
(Biotechnology)  --  are  required  to  search  the  shoes  on  every  application.   Other 
examiners  reported  that  they  continue  to  use  the  shoes  because  of  limitations  in  APS, 
particularly  the  absence  of  pre-1971  text. 


1Overall  levels  of  usage  are  depicted  on  the  charts  on  the  next  page. 


33 


APS  Usage 


1 1   Amount  of  usage  of  the  shoes 


31    Amount  of  usage  of  image  retrieval  through  an 
image  workstation 

1 9    Amount  of  usage  of  text  search  through  an 
image  workstation 

38    Amount  of  potential  usage  of  commercial 
databases 

15    Amount  of  usaqe  of  text  search  through  a  text   W"-'''f:- 
terminal  i 

40a    Amount  of  usage  of  CAS/STN 


40c    Amount  of  usage  of  Dialog   pUp 

40b    Amount  of  usage  of  Derwent 

40g    Amount  of  usage  of  Other 

40f    Amount  of  usage  of  Orbit 

40d    Amount  of  usage  of  Lexis 

40e    Amount  of  usage  of  Westlaw 


■  Heavy  Usage 
13  Light  Usage 


120 
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For  many  veteran  examiners,  the  hand-written  marginalia  scribbled  across  the 
face  sheet  of  the  patent  documents  in  the  shoes  has  become  a  valuable  record  and 
cross-referencing  tool  that  was  never  transferred  to  APS.     Examiners  who  participated 
in  the  focus  groups  also  commented  about  the  value  of  being  able  to  associate  a 
given  patent  with  a  spatial  recollection  of  where  the  patent  was  physically  located. 
Those  examiners  who  tended  to  work  "odd  hours"  liked  the  perpetual  availability  of  the 
shoes  which  are  accessible  no  matter  the  time  of  day  or  night.    For  examiners  who 
routinely  apply  certain  "art  dependent"  terms  -  such  as  molecular  and  chemical 
structures,  mathematical  formulas,  processes,  and  material  layering  --  the  shoes  are 
essential  because  such  inquiries  are  not  currently  executable  on  APS.    Finally, 
examiners  praise  the  shoes  as  a  vehicle  for  allowing  them  to  keep  up  with  the 
technology  in  their  art  area  by  quickly  shuffling  through  stacks  of  patents  that  might 
interest  them. 

Survey  findings  corroborated  that  there  is  still  almost  universal  usage  of  the 
shoes.   Almost  all  examiners  (98%)  have  access  to  the  shoes  and  four-fifths  (81%) 
use  this  research  channel  on  virtually  every  application.   Over  four-fifths  (83%) 
indicate  they  use  the  shoes  nearly  every  day  or  at  least  one  or  more  times  per  week. 
Most  importantly,  almost  all  (92%)  of  those  examiners  with  access  state  that  the  shoes 
completely,  to  a  large  extent  or  to  some  extent  fulfill  their  information  needs. 

While  this  is  the  highest  showing  of  commitment  for  any  of  the  search  tools 
currently  available  at  PTO,  this  finding  cannot  be  taken  on  its  face  as  evidence  that 
examiners  prefer  the  shoes  as  their  primary  research  tool.   Even  heavy  users  of  APS 
Text  Search  often  must  also  search  the  shoes  because  the  patents  in  the  automated 
Text  Search  system  only  date  back  to  1971.   Not  insignificantly,  most  of  the  sixteen 
focus  groups  were  asked  to  speak  theoretically  about  what  research  tool  they  would 
keep  if  they  could  keep  only  one:   all  focus  groups  chose  APS. 

However,  continued  and  heavy  dependency  on  the  shoes  is  inevitable  until 
PTO's  automated  search  systems  are  fully  implemented2.   A  major  PTO  effort  will  also 
be  required  to  improve  computer  literacy  and  re-orient  examiners  towards  electronic 
search  processes.    Reduction  in  dependency  on  the  shoes  is  directly  tied  to  the  ability 
of  APS  to  become  more  comprehensive  and  to  include  the  text  of  all  patents  dating 
back  to  1790,  foreign  patents  and  non-patent  literature.  APS  must  also  become  more 
accessible  to  examiners  --  customized  text  and  image  search  available  right  on  the 
desktop.    Moreover,  examiners  must  be  thoroughly  trained  in  how  to  make  the  most  of 
PTO's  automated  search  tools. 


For  all  of  the  reasons  discussed,  it  is  reasonable  to  expect  that  the  shoes  will  be  as  much  a  21st 
century  icon  as  they  were  in  the  19th  and  20th  century. 
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APS  supports  a  substantial  percentage  of  examiners'  information  requirements 

Although  most  examiners  search  the  shoes  on  a  regular  basis,  a  substantial 
percentage  of  PTO's  examining  corps  also  rely  on  automated  search  tools  in  the 
examination  process.    Examiners  were  asked  a  series  of  questions  about  how  often 
they  used  PTO's  automated  search  tools,  what  aspects  of  these  tools  best  satisfied 
their  requirements3  and  which  were  most  important.   The  survey  data  corroborated  the 
extent  to  which  examiners  believe  that  APS  is  an  indispensable  research  tool. 

All  examiners  have  easy  access  to  only  one  automated  search  tool:   the  Text 
Search  capability.    Because  the  Text  Search  system  can  be  accessed  through  three 
different  channels  --  the  shared  text  terminals,  desktop  PCs  or  the  image  workstation 
--  respondents  were  asked  to  rate  these  channels  separately.   Many  examiners 
currently  have  access  to  Text  Search  through  their  desktop  PC  as  well  as  through  an 
image  workstation  or  text  terminal,  and  all  examiners  are  expected  to  have  desktop 
text  access  by  the  end  of  1994. 

Text  Search  through  a  shared  text  terminal  received  the  highest  rating  in  terms 
of  access  of  all  automated  search  tools.   Seventy-three  percent  (73%)  of  the 
examining  corps  have  access  to  this  tool  and  over  one  third  (35%)  of  those  with 
access  reported  that  they  use  it  heavily  (nearly  every  day  or  at  least  one  or  more 
times  per  week).    Half  (50%)  reported  that  this  tool  essentially  fulfills  their  information 
needs.   (Although  these  percentages  do  not  appear  to  be  unusually  high,  these  ratings 
reflect  only  one  means  of  access  to  APS.   Moreover,  this  means  of  access  will  decline 
as  overall  migration  towards  desktop  computers  in  the  PTO  environment  continues.) 


3Chapter  5:    Performance  Indicators  addresses  examiner  satisfaction  with  the  performance  of 
PTO's  automated  search  tools. 
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The  extent  of  migration  toward  PCs  is  underscored  by  the  fact  that  Text  Search 
through  a  desktop  PC  was  the  second  most  popular  means  of  access  to  APS.    Most 
examiners  currently  have  a  PC  at  or  near  their  desktop  (93%)  and  have  desktop 
access  to  APS  Text  Search  (91%).   Most  (93%)  examiners  have  PCs  that  contain  Intel 
386  chips.4   Half  (52%)  of  those  who  currently  have  desktop  access  to  Text  Search 
are  connected  via  the  PTO  Network  (PTOnet).   Less  than  two-fifths  (39%)  are 
connected  via  a  modem.   Focus  group  commentators  reported  that  while  PTO  has 
provided  some  examiners  with  modems,  other  examiners  have  purchased  and 
installed  modems  for  themselves. 

One  of  PTO's  goals  is  to  provide  all  examiners  with  desktop  access  to  APS 
Text  Search  by  the  end  of  1994  --  and  eventually  to  APS  Image  Search.   At  present, 
one-tenth  (11%)  of  the  examining  corps  does  not  currently  have  APS  text  access  at 
the  desktop  --  either  because  they  do  not  have  a  PC  (2%)  or  their  PC  is  not 
connected  to  APS  via  PTOnet  or  a  modem  (9%).   In  addition,  some  quarter  (26%)  of 
those  examiners  with  desktop  access  are  essentially  not  able  to  use  it  because  their 
PC  is  not  hooked  up  to  a  working  printer.   While  these  examiners  can  perform  a 
search  at  the  desktop,  they  must  turn  to  the  shoes,  the  shared  facilities  or  the  Walk- 
Up  Printers  in  order  to  get  an  actual  copy  of  the  patents  relevant  to  their  search. 

Of  the  three  channels  available  to  access  APS  Text  Search,  examiners  gave 
text  search  through  a  desktop  PC  the  highest  rating  in  terms  of  its  ability  to  fulfill  their 
information  requirements.   Over  four-fifths  (83%)  of  those  examiners  who  are  able  to 
access  Text  Search  from  the  desktop  reported  that  this  tool  essentially  meets  their 
information  needs.   Other  responses  from  the  survey  demonstrate  examiners' 
eagerness  to  migrate  to  a  desktop  environment.   Over  two-fifths  (44%)  of  examiners 
with  PC  access  say  they  access  Text  Search  from  their  desktop  PC  on  most 
applications,  as  compared  to  24%  who  have  access  to  and  use  text  search  through  a 
shared  text  terminal  or  the  20%  who  access/use  it  through  an  image  workstation. 
Moreover,  when  asked  to  rate  a  series  of  fifty-five  potential  enhancements  to  APS,  the 
examining  corps  rated  access  to  image  and  text  on  the  desktop  as  their  number  one  priority. 


4Most  of  the  examining  corps'  desktop  PCs  use  an  Intel  386  microprocessor  chip.   The 
microprocessor  chip  contains  the  central  processing  unit  (CPU)  -  a  computer's  control  center.   The 
CPU  is  a  highly  complex,  extensive  set  of  electronic  circuitry  that  executes  stored  program  instructions 
and  converts  data  input  into  information  output.  The  size  and  composition  of  the  microprocessor  chip 
determines  to  a  great  extent  the  power  and  flexibility  of  the  computer.   The  386  chip  replaced  the  older 
286  chip  and  allowed  users  to  run  several  software  programs  at  the  same  time.   An  even  faster  and 
more  powerful  chip,  the  486,  has  become  the  industry  standard. 

According  to  the  survey  findings,  five  percent  (5%)  of  those  examiners  with  a  desktop  PC  currently  have 
a  286  computer;  most  (93%)  have  a  386;  and,  two  percent  (2%)  have  a  486.    Less  than  1%  of  the 
corps  uses  a  Macintosh  computer.  As  new  features  and  software  capabilities  are  added  to  the  desktop 
workstations,  PTO  will  need  to  upgrade  this  computer  hardware  --  by  bringing  on  more  486  computers  -- 
to  support  advanced  applications. 
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By  the  end  of  1994,  all  examiners  were  scheduled  to  have  access  to  APS  Text 
Search  via  the  PTO  Network  on  their  desktop  PC.   Through  the  desktop  deployment 
project  --  scheduled  to  begin  in  FY1996  and  to  end  in  FY2000  --  all  examiners  will  be 
provided  with  immediate  access  to  image  and  text  search  and  retrieval,  as  well  as  to 
office  automation  services  and  global  patents.   The  installation  of  upgraded  group 
printers  will  also  be  part  of  this  project. 

The  final  Text  Search  channel  available  to  examiners  is  through  the  image 
workstation.   This  channel  received  the  lowest  score  in  terms  of  access  ~  twenty-two 
percent  (22%)  of  the  examining  corps  currently  has  this  capability.   Although  just 
under  three-fifths  (57%)  of  these  examiners  use  it  regularly  (nearly  every  day  or  one  or 
more  times  per  week),  three-quarters  (75%)  say  that  it  basically  fills  their  information 
requirements.   One-fifth  (20%)  of  examiners  with  access  to  it,  use  text  search  at  an 
image  workstation  on  most  applications. 

The  only  way  examiners  can  employ  APS's  Image  Search  capability  is  through 
the  image  workstations.   As  indicated  above,  only  one-fifth  (22%)  of  the  corps 
currently  have  easy  access.   Almost  two-thirds  (63%)  of  the  examiners  who  have 
image  retrieval  access  say  they  use  it  heavily  --  nearly  every  day  or  one  or  more  times 
per  week.   Over  one-quarter  (27%)  use  it  on  most  applications.   Over  three-quarters 
(78%)  indicate  that  image  retrieval  basically  meets  their  information  needs. 

Despite  some  reported  shortcomings  with  the  current  image  retrieval  system, 
examiners  say  they  want  and  need  access  to  the  images  of  patents  dating  back  to 
1790.    Focus  group  commentators  who  did  not  have  access  to  the  image  search 
system  eagerly  expressed  their  interest  in  this  technology  and  the  statistical  findings 
confirmed  the  anecdotal  conclusions.   Of  those  examiners  who  are  not  currently  able 
to  perform  image  retrieval,  over  two-thirds  (69%)  indicated  that  they  want  this 
capability.   The  ability  to  access  image  and  text  at  the  desktop  -  and  to  easily  switch 
between  them  --  was  rated  as  PTO's  top  priority  by  the  examining  corps  (78%). 
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Since  initially  launching  the  image  retrieval  system  in  1988,  PTO  has  always 
planned  to  make  image  search  available  to  all  examiners.5   Examiners  in  groups  1200, 
2100,  2200  and  2300  currently  share  100  dual-screen,  custom-designed  first  and 
second  generation  workstations  that  allow  them  to  access  the  image  database  using 
Classified  Search  and  Image  Retrieval  (CSIR)  software.   Some  forty-eight  (48)  third 
generation  workstations6  and  eight  (8)  new  group  printers  are  scheduled  to  be 
delivered  by  March  13,  1995.   These  workstations  will  be  distributed  between  Groups 
2500  and  2600.    Unlike  the  first  and  second  generation  workstations,  the  third 
generation  workstations  are  a  hybrid  of  custom-designed  and  commercially  available 
technology. 

PTO  is  preparing  to  launch  the  next  phase  of  image  retrieval  capabilities.   An 
internal  discussion  is  underway  about  what  technology  is  best  suited  to  deliver  image 
and  text  to  the  desktop  of  every  examiner.    New  storage  technologies  are  constantly 
being  examined.   A  decision  has  yet  to  be  made  on  whether  the  terminals  need  to 
have  two  screens  or  whether  one  screen  would  be  sufficient.    PTO  expects  to  begin 
supplying  the  new  desktop  workstations  to  the  examining  corps  in  FY1996  and  to 
complete  this  process  by  the  end  of  the  decade. 

Commercial  databases  are  an  important  research  tool 

There  are  a  number  of  commercial  databases  --  notably  CAS/STN,  Dialog, 
Orbit,  Derwent,  Lexis,  Westlaw  --  that  have  been  made  available  to  certain  groups 
within  the  examining  corps.    If  examiners  have  the  proper  account  number,  they  can 
access  databases  that  have  been  "offered"  to  them  at  several  locations:   from  their 
desktop  PC,  from  a  shared  facility  workstation,  or  at  a  terminal  in  the  Scientific  and 
Technical  Information  Center  (STIC).   Use  of  these  commercial  databases  seems  to 
be  growing,  or,  as  one  examiner  put  it: 

With  the  explosion  of  knowledge  in  certain  art  areas,  patent  examination 
has  become  less  dependent  on  searching  the  actual  patents  themselves, 
and  more  dependent  on  being  able  to  search  a  wide  range  of  databases 
that  collectively  describe  the  current  state  of  the  art  [including 
commercial  databases  and  technical  literature].   APS  has  a  long  way  to 
go  in  providing  that  kind  of  encyclopedic,  integrated  capability. 


In  theory,  all  examiners  can  use  the  image  workstations  should  they  chose  to  do  so.  In  reality, 
however,  most  examiners  are  not  trained  to  use  this  search  tool.  In  addition,  the  image  workstations 
are  located  only  in  certain  examining  group  areas. 

6These  new  third  generation  workstations  are  called  shared  use  workstations  or  SUS. 
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This  examiner's  critique  alludes  to  the  fact  that  these  databases  are  essentially 
separate  from  APS.   They  all  require  their  own  log-in/log-off  procedures.   Their 
command  structures  are  inconsistent  and  sometimes  difficult  to  learn.    Most 
importantly,  APS  currently  provides  little  or  no  basis  for  merging  the  results  of  a 
search  conducted  across  several  of  these  databases.7 

While  over  half  (54%)  of  the  examining  corps  does  not  currently  have  access8 
to  any  commercial  databases,  nearly  three-fifths  (59%)  of  these  examiners  want  this 
capability  and  over  one-third  (36%)  say  they  would  use  it  regularly  (nearly  every  day 
or  one  or  more  times  per  week). 

In  some  areas,  particularly  in  the  chemical  and  biotechnology  examining 
groups,  pre-searchers  are  made  available  to  examiners  to  conduct  commercial 
database  searches.   According  to  PTO  officials,  there  are  two  primary  reasons  for  this: 
1)  It  provides  examiners  in  these  areas  with  more  time  to  examine;  and,  2)  Because  of 
the  complexity  of  the  types  of  commercial  databases  available  at  PTO,  it  is  more 
prudent  to  let  experts  use  the  databases  in  order  to  minimize  time  (and  thus  cost) 
logged  on  to  the  databases. 

PTO's  examining  corps  is  a  heterogeneous  professional  community 
with  different  information  needs  and  skill  levels 

Both  focus  group  comments  and  survey  findings  illustrate  that  examiners  have 
strikingly  different  needs  for  electronic  tools.   Focus  group  sessions,  which  included 
participants  from  all  sixteen  examining  groups,  confirmed  that  there  are  significant 
differences  in  the  information  needs,  work  methods,  and  perceptions  about  how  APS 
is  impacting  the  search  process  in  various  art  groups.    Not  only  is  the  very  nature  of 
the  search  process  different,  but  examiners  bring  to  the  job  their  individual  work  habits 
and  familiarity  with  electronic  tools.   These  differences  --  both  individual  and  art  group 
--  have  enormous  implications  for  how  PTO  should  develop  information  tools  to 
facilitate  the  work  of  the  patent  examining  corps. 


One  examiner  in  Group  2100  has  designed  a  program  called  Document  List  Manager  that,  to  a 
certain  degree,  allows  an  examiner  to  duplicate  a  search  across  the  spectrum  of  electronic  information 
resources  available  -  from  APS  to  examiner  digests,  to  the  array  of  sanctioned  commercial  databases. 
While  2100  is  the  only  group  currently  approved  to  use  "Doc  List  Manager,"  other  PTO  organizations 
have  shown  a  keen  interest  in  the  program's  Janus  feature  -  which  pulls  up  every  patent  that  is  cited 
on  a  "good  reference"  (the  past)  as  well  as  every  patent  that  cites  that  same  reference  (the  future)  - 
and  it  may  be  included  in  the  Examination  Toolkit  project. 

Access  to  commercial  databases  is  influenced  by  many  factors  --  the  most  overwhelming  being 
cost.   However,  for  various  personal  reasons,  many  examiners  are  not  attracted  to  this  research  tool. 
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The  extent  of  these  differences  is  dramatized  by  the  fact  that  while  nine  out  of 
every  ten  examiners  working  in  the  mechanical  groups  find  that  the  shoes  fulfill  their 
information  needs,  only  half  of  examiners  in  the  chemical  groups  find  that  to  be  the 
case.    Not  surprisingly,  mechanical  group  respondents  (78%)  were  satisfied  with  the 
speed  with  which  they  could  search  the  shoes,  while  only  sixty-five  percent  (65%)  of 
chemical  group  respondents  were  so  satisfied. 

One-third  of  the  examining  corps  considers  themselves  to  be  computer  novices 

During  the  focus  group  sessions,  examiners  described  themselves  as  having  a 
wide  variety  of  computer  skills.   One  senior  examiner  revealed  a  colleague  had 
provided  him  with  a  diskette  that  automates  all  log-in  procedures  and  brings  him 
directly  to  the  search  prompt.   Without  this  diskette,  he  reported,  he  would  not  be  able 
to  use  APS  at  all.   Other  examiners  demonstrated  a  high  level  of  computer 
sophistication  as  they  described  how  they  overcame  limitations  in  APS's  search 
structure  or  used  advanced  software  applications  in  order  to  make  APS  more  user- 
friendly. 

The  current  level  of  computer  expertise  among  examiners  is  a  serious 
deficiency  that  limits  the  ability  of  the  corps  to  fully  maximize  APS's  power.   Almost 
one-third  (30%)  of  the  examining  corps  described  themselves  as  a  complete  beginner 
or  a  novice  who  still  had  much  to  learn.   Almost  half  of  the  corps  (44%)  rated 
themselves  as  intermediate  users.   Only  one-fourth  (27%)  consider  themselves  to  be 
an  advanced  user  or  computer  pro. 

The  extent  to  which  examiners  are  computer  literate  influences  their  ability  to 
grasp  and  retain  what  they  learn  in  training  sessions;  their  readiness  to  use  electronic 
tools  on  a  regular  basis  in  the  search  process;  their  dependence  on  the  shoes;  their 
level  of  tolerance  when  APS  malfunctions;  and  their  overall  criticisms  of  the 
information  tools  available  to  them  at  PTO. 

Dependence  on  search  tools  is  related  to  the  types  of  applications  examined 

Dependence  on  APS  is  also  related  to  the  types  of  applications  examined  by 
each  of  the  patent  examining  corps'  examining  groups  and  art  units.   The  corps  is 
divided  into  sixteen  broad  examining  groups  --  five  groups  examine  chemical-related 
patent  applications,  six  examine  electrical-related  applications,  and  five  examine 
mechanical-related  applications.   Each  of  these  larger  groups  is  further  broken  down 
into  four  to  sixteen  subgroups,  or  art  units.   According  to  our  survey  findings,  thirty-two 
percent  (32%)  of  the  corps  consider  their  art  to  be  predominantly  chemical,  thirty-five 
percent  (35%)  electrical  and  thirty-three  percent  (33%)  mechanical. 
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Each  of  these  examining  groups  and  art  units  have  different  needs  in  terms  of 
the  types  of  searches  required  and  the  types  of  searches  that  work  best.  Focus  group 
participants  said  that  shoe  search  is  "mandatory"  for  some  groups,  while  others  must 
perform  APS  text  or  image  searches.   They  also  emphasized  that  in  certain  arts,  such 
as  biotechnology  and  information  processing,  examiners  use  APS  on  virtually  every 
case.    Examiners  in  other  arts,  particularly  mechanical,  may  use  APS  only  once  or 
twice  per  month.   The  statistical  data  confirms  the  observations  made  by  focus  group 
commentators. 

Almost  all  mechanical  examiners  (93%)  use  the  shoes  on  a  regular  basis, 
compared  to  just  over  three-quarters  of  chemical  (76%)  and  electrical  (79%) 
examiners.   Most  mechanical  (98%)  and  electrical  (92%)  examiners  indicate  that  the 
shoes  essentially  fulfill  their  information  needs  and  over  four-fifths  of  chemical 
examiners  (86%)  are  as  satisfied  with  the  shoes  in  terms  of  meeting  their  information 
requirements. 

In  addition,  mechanical  examiners  indicated  that  they  use  APS  and  other 
automated  search  tools  on  a  less  frequent  basis  than  chemical  and  electrical 
examiners.    For  example,  two  out  of  ten  mechanical  examiners  stated  that  they  use 
text  search  through  a  shared  terminal  on  a  regular  basis,  while  four  out  of  ten 
electrical  and  five  out  of  ten  chemical  examiners  use  it  regularly.   Three-quarters 
(76%)  of  the  electrical  examiners  and  half  (48%)  of  the  chemical  examiners  use  image 
search  on  a  regular  basis,  compared  to  two-fifths  (39%)  of  the  mechanical  examiners. 
Seven  out  of  ten  mechanical  examiners  state  that  text  search  through  their  desktop 
PC  fulfills  the  information  needs  of  their  job,  compared  to  nine  out  of  ten  chemical 
examiners.   And,  almost  no  mechanical  examiners  indicated  that  they  use  any  of  the 
commercial  databases  on  a  regular  basis. 
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Chapter  4:     ACCESS  TO  RESEARCH  TOOLS 


Examiners  need  comprehensive  and  consistent  access  to  PTO's 
automated  search  tools 

During  the  focus  group  sessions  and  interviews,  examiners  and  managers  alike 
identified  a  critical  measurement  of  the  effectiveness  of  APS  and  other  automated 
search  tools  --  to  what  extent  can  examiners  actually  use  these  capabilities?  Access 
to  these  tools  is  essential  if  examiners  are  expected  to  employ  them  on  a  regular 
basis  in  the  examination  process. 

Access  to  date,  however,  is  neither  comprehensive  nor  consistent.   Some 
examiners  have  access  to  Image  Search,  others  do  not.   Some  access  Text  Search 
through  a  shared  terminal,  others  through  a  PC,  and  still  others  through  an  image 
workstation.    Some  examiners  are  able  to  search  commercial  databases  and  to  take 
advantage  of  graphical  user  interfaces  on  the  PC;  other  examiners  either  lack  access 
to  these  tools  or  just  do  not  know  how  to  use  them.   In  addition,  focus  group 
commentators  reported  problems  with  log-in  IDs  and  passwords  and  maintenance  of 
terminals  and  printers.   Clearly,  examiners  cannot  use  PTO's  automated  search  tools 
and  related  equipment  if  they  do  not  have  access  to  these  resources  or  if,  for  one 
reason  or  another,  they  just  are  not  available  when  examiners  want  to  use  them. 

Examiners  express  dissatisfaction  with  their  access  to  PTO's  automated  tools 

Examiners  were  asked  a  number  of  questions  throughout  the  survey  in  order  to 
clarify  the  access  issue.    In  the  first  section  --  overall  information  resource  assessment 
--  examiners  were  asked  one  direct  access  question  and  two  related  questions.   The 
responses  to  these  questions  are  depicted  on  the  following  page.   Just  over  one-third 
(34%)  of  examiners  expressed  their  dissatisfaction  with  their  "access  (hook-up)  to  APS 
and  other  databases  needed  to  do  (their)  job."  This  is  particularly  true  of  mechanical 
examiners,  almost  half  (44%)  of  whom  indicated  dissatisfaction  with  their  access  to 
automated  tools.   Examiners  also  expressed  dissatisfaction  with  the  availability  of 
equipment  needed  to  access  PTO's  automated  tools  --  terminals,  PCs,  CRTs,  and 
printers.   Thirty  percent  (30%)  of  the  examining  corps  indicated  that  they  are  generally 
or  very  dissatisfied  with  the  "availability  and  maintenance  of  terminals,  PCs,  and  CRTs 
to  do  (their)  job."  Almost  two-fifths  (37%)  expressed  dissatisfaction  with  the  availability 
of  printers. 
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Examiners'  access  to  PTO's  automated  tools  is  varied 

Examiners  were  also  asked  to  indicate  whether  or  not  they  had  access  to  each 
of  the  primary  --  automated  and  non-automated  --  search  tools  available  at  PTO:   the 
shoes,  text  search  through  a  shared  terminal,  text  search  through  an  image 
workstation,  text  search  through  a  desktop  PC,  image  search  at  an  image  workstation, 
and  commercial  databases.   The  data  (illustrated  below)  indicates  that  while  almost 
the  entire  examining  corps  has  access  to  the  shoes,  access  to  the  other  channels  -- 
all  of  which  are  automated  --  is  varied. 

The  entire  examining  corps  has  access  to  APS  Text  Search.  However,  not  all 
Text-accessible  channels  are  available  to  all  examiners.   Seven  out  of  every  ten 
examiners  said  they  have  access  through  a  shared,  single-screen  text  terminal.   Two 
out  of  ten  said  they  have  access  to  Text  Search  through  an  image  workstation.    Nine 
out  of  ten  examiners  said  they  currently  have  access  to  Text  Search  through  a 
desktop  PC. 

Less  than  one-quarter  (22%)  of  the  examining  corps  currently  has  access  to 
APS  Image  Search.   Just  under  three-fourths  (69%)  of  examiners  --  spread  fairly 
evenly  among  chemical,  electrical  and  mechanical  examiners  -  who  currently  don't 
have  access  to  an  image  workstation  indicated  they  would  like  to  be  able  to  use  this 
critical  search  tool.   Given  the  fact  that  image  retrieval  at  the  desktop  -  through  the 
desktop  deployment  project  --  will  not  be  available  to  all  examiners  until  the  end  of  the 
decade,  demand  for  access  to  the  image  workstation  will  surely  increase  over  time  as 
more  examiners  become  exposed  to  its  capabilities. 

Access  to  Search  Tools 


1 0.  Access  to  the  shoes 


24  Access  to  APS  Text  through  desktop  PC 


1 4  Access  to  APS  Text  through  shared  text 
terminal 


1  d  Overall  access  to  APS  &  other  databases 

36  Access  to  commercial  databases 

1 8  Access  to  APS  Text  through  image  workstation 

29  Access  to  APS  Image  through  image  workstation 
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Over  half  (54%)  of  the  examining  corps  does  not  currently  have  access  to  any 
commercial  databases  and  three-fifths  of  these  examiners  indicate  that  they  want 
access  to  this  search  tool.1   Chemical  examiners  currently  have  tjne  greatest  access  to 
commercial  databases  and  the  strongest  interest  in  acquiring  more  access.   While 
mechanical  examiners  have  the  lowest  access  rate  (17%)  for  commercial  databases, 
just  under  half  (47%)  of  those  who  do  not  currently  have  access  want  to  be  able  to 
use  these  tools. 

Examiners'  access  to  these  search  tools  is  also  impacted  by  the  availability  of 
automated  systems  at  any  given  time.   Terminals  used  to  access  these  systems  may 
be  in  use  or  out  of  order.   As  depicted  in  the  chart  below,  examiners  are  most 
satisfied  with  the  availability  of  the  image  workstations  --  not  surprising  since  less  than 
a  quarter  (22%)  of  the  examining  corps  currently  has  access  to  this  equipment. 
Chapter  5:    Performance  Indicators  further  explores  this  issue  of  availability. 


AVAILABILITY 


%  Satisfaction 


(Rank  Order  by  Total  Corps) 


| BB All  Examiners  CDChemical  only  E3 Electrical  only  □Mechanical  only 


1Some  commentators  we  spoke  to  indicated  that  the  cost  of  connect  time  and  system  usage  for 
commercial  databases  limits  or  restricts  access  to  these  systems  for  many  examiners. 
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Complete  APS  Text  Search  access  through  PCs  is  impacted  by  other  factors 

Many  examiners  who  participated  in  the  focus  group  sessions  claimed  that  they 
could  not  access  APS  through  their  desktop  PC  because  the  appropriate  management 
organization  did  not  have  enough  (constant)  log-in  IDs  and/or  (changeable)  passwords 
to  give  to  all  examiners.   They  were  unable  to  explain  whether  this  perceived 
"shortage"  was  due  to  some  inadequacy  within  APS  itself  (the  mainframe),  to  PTOnet, 
to  both  --  or  to  neither.2 

The  survey  results  indicate  that  as  of  late  October,  1994,  fully  ninety-three 
percent  (93%)  of  the  examining  corps  had  one  or  more  PCs  on  their  desktop.   Three- 
quarters  (74%)  of  those  PCs  were  hooked  up  to  a  working  printer  and  ninety-one 
percent  (91%)  of  those  PCs  had  some  access  to  APS  Text  Search  -  fifty-two  percent 
(52%)  through  PTOnet  and  thirty-nine  percent  (39%)  through  a  modem.   This  means 
that  nine  percent  (9%)  of  all  examiners  with  a  PC  still  have  no  access  to  APS  Text 
Search  in  any  way  --  and  an  additional  thirty-nine  percent  (39%)  can  access  APS  only 
through  slower,  more  trouble-prone  modems. 

Altogether  then,  at  the  time  the  survey  was  fielded,  approximately  half  of  the 
examining  corps  was  not  yet  hooked  up  to  PTOnet  nor  to  any  of  the  office  automation 
potential  that  PTOnet  enables.   Survey  results  should  look  very  different  three  months 
from  now  because  PTO  is  planning  to  provide  desktop  PTOnet  access  to  all 
examiners  by  December  31,  1994.   If  PTO  management's  IRM  schedule  is  followed, 
additional  file  servers  will  be  installed  in  November-December,  1994.   (In  fact,  four  file 
servers  were  deployed  during  the  drafting  of  this  report,  adding  hundreds  of  examiners 
to  the  network.3) 


Interviews  with  PTO  managers  and  CIO  personnel  revealed  that  there  is  no  inherent  limitation  with 
regard  to  the  number  of  access  IDs  for  either  PTOnet  or  the  mainframe. 

3These  additional  file  servers  will  enable  all  examiners  to  go  onto  the  PTO  network  and  eventually 
access  a  full  array  of  office  automation  products,  of  which  APS  Text  Search  is  only  one. 
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Access  to  commercial  databases  is  also  expected  to  grow 

While  focus  group  commentators  expressed  a  growing  need  for  access  to 
commercial  databases  in  the  search  process,  the  data  reveals  a  relatively  low  level  of 
usage  of  this  search  tool.   This  low  usage  rate  could  be  influenced  by  a  couple  of 
factors.   First,  examiners  are  currently  not  able  to  easily  transfer  a  search  strategy 
from  APS  text  or  image  to  a  commercial  database  or  even  between  databases. 
Almost  two-thirds  (63%)  of  the  examining  corps  reported  that  more  consistent 
commands  across  all  databases  used  at  PTO  would  have  a  significant  positive  impact 
on  their  job  performance.  Almost  three-fifths  (59%)  stated  that  the  ability  to  access  a 
variety  of  databases  in  one  session  without  having  to  log  on  and  off  of  each  one 
separately  would  also  be  useful  to  the  examining  corps. 

Secondly,  many  examiners  must  leave  their  immediate  work  area  to  access 
commercial  databases.   Some  three-fifths  (57%)  of  examiners  said  that  the  ability  to 
access  all  commercial  databases  from  their  desktop  PC  would  have  a  significant 
impact  on  their  job  performance.    Examiners  rated  this  as  one  of  PTO's  top  fifteen 
priorities  out  of  a  list  of  fifty-five  potential  enhancements  to  APS. 

PTO  has  already  recognized  the  importance  of  commercial  databases  in  the 
search  process.    In  the  most  recent  PTO  strategic  plan  ~  the  Strategic  Information 
Technology  Plan:   FY1995-FY2000  ~  specific  steps  are  outlined  to  make  this  search 
tool  more  readily  available  and  accessible  to  the  examining  corps.   As  part  of  the 
planned  Examination  Toolkit  project,  PTO  will  deliver  an  enhanced  commercial 
database  access  tool  to  the  examiners'  desktops  by  September,  1995.   This  tool  will 
include  document  manager  features  that  facilitate  the  transfer  of  search  strategies 
across  all  automated  search  tools. 
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Better  communication  about  graphical  user  interfaces  can  make  APS  more 
accessible 

The  Examination  Toolkit  project  is  not  PTO's  first  attempt  to  develop  software 
applications  that  make  aspects  of  APS  more  accessible  to  the  examiner.   Within  the 
last  year,  PTO's  Information  System  division  created  communication  software  that 
included  graphical  user  interfaces  (GUI)  --  WinCom  (the  hard  disk  interface)  and 
WinNet  (the  PTO  Network  interface)  --  which  allow  examiners  to  operate  APS  at  the 
desktop  in  a  more  user-friendly,  Windows-like  environment.4   Focus  group 
commentators  reported  that  PTO  had  done  a  poor  job  of  both  communicating  the 
availability  of  these  GUIs  as  well  as  training  examiners  to  use  these  tools,  and  the 
statistical  data  gathered  in  the  survey  confirmed  their  observations. 

Survey  findings  reveal  that  while  almost  half  (46%)  of  those  examiners  with 
desktop  APS  access  are  using  WinNet,  only  one-tenth  (14%)  are  using  WinCom. 
Less  than  one-fifth  (16%)  are  not  using  either  WinCom  or  WinNet  and  an  amazing 
quarter  (24%)  are  not  sure  if  they  are  using  WinCom  or  WinNet.   These  findings 
compare  with  focus  group  sessions,  during  which  commentators  revealed  that  many 
examiners  are  not  aware  of  either  the  value  or  availability  of  these  GUI.    Other 
examiners  prefer  non-CIO-supported  products,  such  as  PROCOM,  that  they 
sometimes  purchase  and  install  themselves. 

Migration  towards  the  desktop  environment  will  reduce  demand  for  shared  APS 
facilities 

The  need  for  access  to  all  but  one  of  the  APS-dedicated  channels  is  expected 
to  decline  over  the  next  five  years  as  PTO  migrates  its  search  systems  to  a  desktop 
environment.    Shared,  single-screen  text  terminals  are  being  phased  out  as  more 
examiners  become  hooked  up  to  PTOnet  at  their  desktop  PC.   The  final,  third 
generation,  image  workstations  will  be  deployed  to  Groups  2500  and  2600  by  mid- 
March,  1995.    No  more  plans  are  underway  or  envisioned  to  develop  new  or  better 
shared  APS  facilities.   The  new  generation  of  APS-dedicated  automated  tools  will  be 
developed  as  desktop  terminals  which  can  access  Text  Search,  Image  Search,  and 
the  to-be-developed  Global  Patents  and  Non-Patent  Literature  Systems  all  in  a  user- 
friendly,  Windows-based  operating  environment. 


Microsoft's  Windows  product  is  one  of  the  most  popular  graphical  user  interfaces  (GUI)  available 
for  PCs.   Instead  of  having  to  remember  a  command  to  type  to  open  a  program  or  execute  a  function, 
Window  users  make  selections  from  choices  available  on  the  computer  screen.    Users  select  an  option 
by  clicking  on  it  -  whether  it  is  an  icon  or  a  written  function  -  with  a  mouse  button.   Windows  allows 
users  to  multitask  -  run  a  number  of  applications  at  the  same  time.   In  several  windows  on  the 
computer  screen,  a  user  can  pull  up  a  spreadsheet  program,  a  word  processing  program  and  an 
electronic  mail  program. 
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Chapter  5:     PERFORMANCE  INDICATORS 


Examiners  are  committed  to  APS  as  a  productivity  enhancer 

Focus  group  commentators  were  virtually  unanimous  in  their  support  of  APS  as 
a  productivity  enhancer.   APS  currently  enables  examiners  to  search  three 
monumental  databases  of  patents  and  patent  abstracts  --  U.S.  Text,  U.S.  Image  and 
Japanese  Abstracts.   Other  databases  --  more  text,  global  patents,  non-patent 
literature  --  are  expected  to  be  added  to  the  system  in  the  future.    Focus  group 
commentators  predicted  that  dependency  on  APS  will  increase  even  more  as  the  body 
of  patent  and  related  technical  information  expands  and  full  search  capabilities  reach 
the  desktop. 

Although  examiners  may  be  dissatisfied  with  certain  aspects  of  the  tools  which 
allow  them  to  access  APS,  they  clearly  depend  on  these  tools  to  perform  their  job. 
Looking  across-the-board  at  examiners'  ratings  of  the  extent  to  which  any  given  tool 
fulfills  their  information  requirements,  they  expressed  a  high  level  of  commitment  to 
PTO's  automated  search  tools.    Half  (50%)  of  those  with  access  to  text  search  at  a 
shared  use  text  terminal  indicate  that  it  essentially  meets  their  information  needs. 
Three-quarters  say  that  text  search  at  an  image  workstation  (75%)  and  image  search 
at  an  image  workstation  (78%)  meets  their  needs.   And  over  four-fifths  (83%)  indicate 
that  text  search  at  the  desktop  meets  their  needs. 

As  the  number  of  patent  applications  continues  to  increase,  as  well  as  the  body 
of  foreign  patents  and  non-patent  technical  literature,  examiners'  dependency  on  APS 
will  increase  as  the  need  to  sift  through  mounds  of  information  also  rises.    PTO 
received  186,123  patent  applications  in  FY1994  and  this  number  is  expected  to  grow 
to  225,000  by  1999.1    Not  only  are  there  more  applications  to  examine,  there  are  more 
patents  —  both  foreign  and  domestic  -  to  search  and  non-patent  literature  to  review. 
Not  surprisingly,  examiners  rated  the  ability  to  search  foreign  art  abstracts  and 
commercial  databases  from  the  desktop  as  two  of  PTO's  top  fifteen  priorities  out  of  a 
list  of  fifty-five. 


1File  integrity  is  widely  perceived  as  APS's  strongest  suit.   Examiners  particularly  welcome  APS's 
ability  to  search  millions  of  domestic  patents  with  almost  100%  file  integrity  without  having  to  leave  their 
own  work  area.   The  reliability  of  a  shoe  search  is  consistently  compromised  by  the  fact  that  all  patents 
are  not  always  available  to  search  and  review.  One-third  (33%)  of  the  examining  corps  indicated  in  the 
survey  that  they  were  generally  or  very  dissatisfied  with  the  file  integrity  of  the  shoes. 

51 


Despite  examiners'  increasing  reliance  on  the  system,  APS  is  clearly  not 
meeting  some  of  their  requirements.   During  the  focus  groups,  examiners  seemed  to 
be  quietly  frustrated  about  a  system  that  continues  to  disappoint  them  in  some  ways. 
The  recent  survey  allowed  examiners  to  express  their  satisfaction  --  or  dissatisfaction 
--  with  the  performance  of  PTO's  automated  search  tools  and  related  equipment.   This 
chapter  describes  these  performance  indicators  and  includes  data  on  the  network 
which  supports  the  automation  program. 

While  usage  of  APS  is  going  up  dramatically  within  the  examining  corps, 
relevant  data  on  the  potential  impact  on  productivity  is  available  from  multiple 
sources.   Thoughtful  analyses  must  take  many  variables  into  account. 

The  following  question  was  asked  throughout  the  survey  instrument:   "To  what 
extent  does  [this  information  channel]  fulfill  the  information  needs  of  your  job?"  The 
responses  to  this  series  of  questions  yielded  the  following  results: 

(Base:    Examiners  with  access  to  each  specific  channel) 

Percent  "Completely," 
Information  Channel  "To  a  large  extent"  or 

"To  some  extent"  fulfills 
information  needs 

The  shoes  92% 

Text  search  through  a  desktop  PC  83% 

Image  retrieval  through  an  image  workstation  78% 

Text  search  through  an  image  workstation  75% 

Text  search  through  a  shared,  single-screen  text  terminal  50% 


Although  these  results  indicate  that  the  shoes  may  still  be  the  primary  source  of 
patent  information  (particularly  since  pre-1971  text  is  not  available  on  APS),  electronic 
channels  are  an  indispensable  part  of  the  mix.    It  is  only  through  automated  search 
channels  that  examiners  can  "pre-screen"  vast  amounts  of  material  within  specified 
classes  and  sub-classes  and  to  eliminate  duplicates.    In  fact,  "hard  data"  --  in  the  form 
of  monthly  reports  to  senior  management  --  show  usage  of  both  text  search 
(Messenger)  and  image  retrieval  (CSIR)  increasing  by  leaps  and  bounds  within  the 
examining  corps  during  the  past  few  years,  even  though  the  number  of  examiners  has 
increased  only  slightly: 
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Approximately  209,000  total  session  hours  [on  all  PTO  search 
applications]  were  logged  by  examiners  in  FY93,  which  was  19%  greater 
than  FY92.  (Information  Systems  Executive  Report,  September,  1994, 
Page  ES.2) 

A  comparison  of  the  last  12  months  to  the  same  period  of  the  previous 
year  shows  Messenger  hours  have  increased  30%,  CSIR  hours  up  51% 
[these  statistics  reflect  mostly  examiner  users,  but  also  public  users  and 
"other"  users].   (Information  Systems  Executive  Report,  September, 
1994,  Page  ES.2) 

The  peak  number  of  examiner  session  hours  on  Messenger  occurred  in 
September  1994  with  15,088  hours  logged.   There  were  95,046  session 
hours  logged  by  examiners  during  FY93  which  was  14%  greater  than 
FY92.    Session  hours  for  FY94  are  up  30%.   (Information  Systems 
Executive  Report,  September,  1994,  Page  1.2) 

Patent  examiners  currently  account  for  74%  of  all  Messenger  text 
searching.   The  number  of  examiners  has  remained  relatively  stable, 
showing  an  annual  growth  rate  of  approximately  5%.   There  was  an 
average  of  1,390  examiners  in  FY92,  increasing  to  1,482  in  FY93. 
During  the  last  12  months  an  average  of  1,546  examiners  have  used 
Messenger  which  is  up  by  4%  over  the  previous  12  months.   (Information 
Systems  Executive  Report,  September,  1994,  Page  1.2) 

A  comparison  of  the  last  12  months  against  the  previous  twelve  months 
shows  a  51%  increase  in  CSIR  session  hours  by  all  users  and  an 
increase  of  36%  by  examiners.   (Information  Systems  Executive  Report, 
September,  1994,  Page  2.2) 

During  FY93  examiners  logged  67,878  hours  on  CSIR  which  was  55% 
greater  than  FY92.   Session  hours  have  increased  by  51%  over  the  last 
12  month  period.   (Information  Systems  Executive  Report,  September, 
1994,  Page  2.2) 


53 


Drawing  straightforward  conclusions  from  this  available  "hard"  data  regarding 
the  overall  productivity  impact  of  APS  requires  an  understanding  of  both  the  unique 
requirements  of  the  system  and  the  many  factors  that  must  come  into  play.2   For 
example,  PTO's  annual  reports  show  that  total  applications  filed  per  year3  went  up 
fifteen  percent  (15%)  between  1989  and  1993:4 

1989  1990  1991  1992  1993 


163,306  174,711  178,083  185,446  188,099 

Pendency  time  of  average  patent  application,  in  months,  went  up  six  percent 
(6%)  in  this  period:5  (Note  that  during  the  same  period,  total  applications'filed 
increased  by  fifteen  percent  (15%).) 

1989  1990  1991  1992  1993 


18.4  18.3  18.2  19.1  19.5 

In  addition,  the  number  of  appeal  cases  filed  per  year  went  up  seventeen 
percent  (17%):6 

1989  1990  1991  1992  1993 


3,829  3,356  3,649  4,281  4,487 


2lt  may  be  useful  to  recall  that  ten  years  after  the  introduction  of  the  first  PCs,  management  experts 
were  hard  pressed  to  find  major,  measurable  impact  on  the  productivity  of  the  U.S.  economy  -  yet  no 
one  could  conceive  of  life  without  the  PC.   One  suspects  that  either  the  "measurement"  of  productivity 
was  inadequate,  or  our  way  of  doing  work  has  been  qualitatively  -  if  not  quantitatively  -  changed.   The 
issue  of  "productivity  impact"  as  traditionally  measured  is  therefore  somewhat  moot. 

3These  figures  include  Design  patents. 

4U.S.  Patent  and  Trademark  Office,  Annual  Report,  FY1993,  page  53. 

5U.S.  Patent  and  Trademark  Office,  Annual  Report,  FY1993,  page  53. 

6U.S.  Patent  and  Trademark  Office,  Annual  Report,  FY89,  page  57.   U.S.  Patent  and  Trademark 
Office,  Annual  Report,  FY90,  page  62.   U.S.  Patent  and  Trademark  Office,  Annual  Report,  FY91 ,  page 
65.   U.S.  Patent  and  Trademark  Office,  Annual  Report,  FY92,  page  60.    U.S.  Patent  and  Trademark 
Office,  Annual  Report,  FY93,  page  57. 
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During  the  same  period,  the  number  of  decisions  affirmed  actually  declined  by 
nineteen  percent  (19%)  --  a  statistic  that  must  be  balanced  by  the  fact  that  the  total 
number  of  patent  decisions  rendered  increased  by  the  same  percentage  (19%). 
Although  the  size  of  the  examining  corps  increased  --  as  the  table  below  illustrates  -- 
the  age  and  experience  levels  of  the  examiners  was  trending  downward  and  their  APS 
and  other  automated  tool  training  opportunities  were  limited  at  best. 

During  that  period,  the  number  of  full-year  patent  examiners  and  SPEs 
(excluding  detailees)  went  up  twenty-eight  percent  (28%):7 


1989 


1990 


1991 


1992 


1993 


1,445 


1,472 


1,796 


1,768 


1,851 


Informed  PTO  commentators  say  that  the  length  and  complexity  of  the  average 
patent  has  increased  between  1989  and  1993.   Appeals  decisions  by  applicants  can 
be  made  for  a  variety  of  reasons,  including  the  applicant's  desire  to  delay  publication 
and,  hence,  dissemination  of  his/her  patented  information  to  the  public.   This  decision 
has  nothing  to  do  with  the  quality  (or  lack  thereof)  of  an  examiner's  decision. 

Many  focus  group  participants  observed  that  electronic  search  and  retrieval 
offers  the  only  hope  of  staying  on  top  of  the  expanding  base  of  patent  and  non-patent 
literature.   Likewise,  electronic  search  and  retrieval  offers  the  only  means  of  serving 
an  ever-expanding  number  of  "outside"  users  (domestic  and  foreign)  and  satisfying  the 
increasing  demand  for  patent  copy  sales. 

Therefore  the  question  is  not  whether  to  go  forward  with  APS  or  "back  to  the 
shoes,"  but  how  to  further  fine  tune  APS  to  the  needs  of  the  various  user  communities 
in  order  to  increase  its  impact  on  the  way  that  work  gets  done.8 

Survey  data  indicate  that  equipment  availability  and  maintenance  is  acceptable 
in  image  workstation  areas,  not  acceptable  elsewhere 

Many  focus  group  commentators  said  that  on  any  given  day,  many  of  the 
terminals  or  printers  could  be  out  of  commission  in  the  shared  facility  workspaces,  and 
those  problem  cases  are  said  to  sometimes  go  unattended  for  days  on  end.   Others, 
however,  tended  to  dispute  such  claims,  and  cite  data  that  suggests  good 
maintenance  of  the  equipment  and  prompt  response  to  problems  reported. 


Patent  Production  Model 


A  number  of  graphs  derived  from  PTO  reports  pictorially  depict  illustrative  trend  data  on  volume  of 
applications,  pendency  and  appeals.   See  Appendix  D. 
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The  survey  produced  the  following  "acceptance  scores"9  concerning  the 
availability  and  maintenance  of  equipment  in  PTO's  search  and  retrieval  workspaces: 


Text  search  through 

a  shared  text 

terminal 

Text  search  through 

an  image 

workstation 

image  retrieval 

through  an  image 

workstation 

Availability  of  a  terminal 
when  you  need  to  use  it 

79% 

91% 

89% 

Operability  and 
maintenance  of  terminals 
1       and  CRTs  in  the  shared 
facility  workspace 

61% 

71% 

76% 

Operability  and 

maintenance  of  printers 

in  the  shared  facility 

workspace 

47% 

79% 

Not  asked 

As  the  chart  illustrates,  the  acceptance  score  for  availability  of  terminals  in  the 
image  workstation  areas  is  quite  high  (91%/89%),  while  the  corresponding  score  for 
the  shared  text  search  areas  is  significantly  lower  (79%).   The  acceptance  score  for 
operability  and  maintenance  of  terminals  and  CRTs  in  the  image  workstation  area  is 
lukewarm  (71%/76%),  and  again  the  corresponding  score  for  the  shared  text  search 
areas  is  significantly  lower  (61%).   The  acceptance  score  for  operability  and 
maintenance  of  printers  in  the  image  workstation  areas  is  quite  respectable  (79%), 
while  the  corresponding  score  for  the  shared  text  search  areas  is  exceptionally  poor 
(47%). 

The  latter  findings  can  be  summarized  in  descending  order  as  follows: 


Issue 


Acceptance  Score 


Availability  of  an  image  workstation  when  needed 

Operability  and  maintenance  of  printers  in  image 
workstation  workspaces 

Availability  of  shared  text  search  terminal  when  needed 

Operability  and  maintenance  of  terminals  and  CRTs 
in  image  workstation  workspaces 

Operability  and  maintenance  of  terminals  and  CRTs 
in  shared  text  search  workspaces 

Operability  and  maintenance  of  printers  in  shared  text  search  workspaces 


90%  (avg) 

79% 
79% 

74%  (avg) 

61% 
47% 


The  figures  for  "text  search  through  an  image  workstation"  and  "image  retrieval  through  an  image 
workstation"  are  virtually  identical  —  which  is  what  one  would  expect,  since  these  ratings  pertain  to  the 
same  "image  workstation"  workspaces.   These  figures  are  accurately  reflecting  the  perceptions  of  the 
examining  corps  across  the  broader  range  of  items. 
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These  results  suggest  a  "two-tiered"  situation  has  developed  within  PTO: 
image  workstations  are  in  good  supply  and  are  relatively  well-maintained  for  the 
handful  of  groups  that  currently  use  them.    However,  only  one-fifth  (22%)  of  the  patent 
corps  have  access  to  image  workstations.   The  seventy-three  percent  (73%)  of  the 
corps  that  have  access  to  shared  text  terminals  find  those  terminals  less  available  and 
less  well  maintained. 

Probably  the  best  available  "hard"  data  on  the  availability  of  APS  equipment  is 
the  "Statistical  Overview  on  APS  Usage"  published  monthly  by  the  Search  and 
Information  Resources  Administration.   One  series  of  graphs  in  that  report  shows  the 
maximum  number  of  (image)  workstations  that  were  in  use  at  one  time  each  day,  for 
each  of  the  four  current  user  groups.10 

In  Group  1200,  for  example,  there  were  no  more  than  six  image  workstations 
(25%)  in  use  at  one  time  on  a  couple  of  days  during  the  month  of  August,  1994,  while 
there  are  a  total  of  twenty-four  workstations  in  that  group.    (One  commentator  said 
that  Group  1200  got  the  workstations  fairly  recently,  and  is  not  yet  "in  business.") 

On  the  other  hand,  Group  2100  shows  a  maximum  of  twenty-three  workstations 
(79%)  in  use  at  one  time  on  a  couple  of  days  during  the  month  of  August,  1994,  while 
there  are  a  total  of  twenty-nine  workstations  in  that  group.   This  suggests  that  the 
overwhelming  percentage  of  workstations  in  that  group  were  available  and  in  good 
working  order. 

The  same  pattern  appears  in  Group  2200  and  Group  2300  --  usage  of  image 
workstations  only  approaches  the  maximum  number  available  on  a  few  days  during 
the  month. 

One  of  the  exhibits  in  this  report  shows  the  number  of  image  workstation 
malfunctions  reported  to  the  Help  Desk  and  requiring  customer  support  intervention. 
The  maximum  number  of  "incidents"  from  the  four  combined  user  groups  (1200,  2100, 
2200,  2300)  never  exceeded  seven  on  any  day  in  August,  1994  (the  current  standards 
threshold),  while  the  number  of  image  workstations  in  these  groups  was  ninety. 

According  to  one  group  director,  availability  of  image  workstations  tends  to  be 
90%  or  better  within  the  current  user  groups.    In  the  opinion  of  that  director  --  who 
currently  heads  an  examining  group  that  doesn't  have  image  access  --  availability  of 
image  workstations  has  never  been  a  significant  problem  in  the  group  he  headed  that 
had  image  access. 


10See  Appendix  D  for  exhibits. 
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However,  examining  groups  with  image  workstations  comprise  just  over  twenty 
percent  (22%)  of  the  patent  examining  corps,  and  there  are  no  comparable  exhibits  in 
the  monthly  Statistical  Overview  that  show  the  availability  of  shared  text  search 
terminals  for  the  remaining  bulk  of  the  corps. 

Availability,  accessibility  and  maintenance  of  printers  are  essential  factors  to 
APS  satisfaction 

Examiners  are  able  to  print  out  text  and  images  from  the  APS  system  from  four 
sources:   local  printers  connected  directly  to  an  image  workstation  or  text  terminal, 
group  printers  networked  to  an  image  workstation  or  text  terminal,  a  walk-up  printer 
that  has  no  search  capabilities  but  can  print  out  any  requested  patent,  and  networked 
printers  that  are  connected  to  desktop  PCs  (these  are  often  located  in  group 
secretarial  stations).   The  printers  available  to  examiners  are  a  mixture  of  laser 
printers  and  dot  matrix  antiques. 

During  the  interview  phase,  examiners  expressed  widespread  criticism  of  printer 
maintenance  and  technical  support.   Some  examiners  claimed  that  the  local,  group 
and  walk-up  printers  are  constantly  running  out  of  paper  or  toner  and  that  the  printers 
are  often  out  of  commission  altogether.   Although  examiners  are  instructed  to  not 
touch  the  printers  and  to  immediately  call  the  Help  Desk  for  printer  support,  the 
unhelpfulness  and,  more  importantly,  untimeliness  of  the  support  offered  has  led  some 
examiners  to  do  their  own  printer  servicing  anyhow. 

The  focus  groups  also  surfaced  two  concerns  about  the  printing  out  of  a 
selected  patent  or  set  of  patents.   The  first  concern  is  that  the  APS  terminal  gives  the 
examiner  no  indication  of  the  number  of  pages  in  a  given  patent.   Once  the  print 
button  is  hit,  there  is  no  easy  way  to  stop  the  process  ~  even  if  it's  an  800  page 
"jumbo"  patent  that  can  potentially  tie  up  the  printer  for  hours.   While  many  examiners 
seemed  happy  to  have  walk-up  printers  available  to  them,  others  complained  about 
the  printer  churning  away  to  produce  "jumbo  patents"  in  their  entirety  while  only  a 
single  page  was  of  value  to  the  search  underway. 

The  second  complaint  stemmed  from  the  "error  processing  report"  notices  that 
are  printed  out  if  a  page  is  not  available.   An  examiner  might  request  a  copy  of  a 
patent  from  one  of  the  walk-up  printer  terminals  but  receive  instead  a  set  of  error 
processing  report  notices  --  one  for  each  page  of  the  patent  that  is  not  available! 
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The  quality  and  maintenance  of  printers  in  the  shared  text  terminal  areas  poses 
problems  for  some 

The  printers  in  the  shared  text  terminal  areas  are  judged  to  be  in  poor  condition 
and  are  sometimes  dot  matrix  antiques.   At  the  same  time,  it  should  be  borne  in  mind 
that  group  printers11  (walk-up  printers)  are  now  producing  the  bulk  of  required  patent 
copies  for  the  examiners,  so  the  poor  condition  of  printers  in  the  shared  text  terminal 
areas  is  perhaps  less  critical.    In  addition,  text  search  through  the  desktop  PC  is  likely 
to  become  the  dominant  channel  within  the  next  twelve  months  --  if  it  is  not  already  -- 
so  future  printers  will  be  needed  near  the  desktop.    In  the  future,  shared  text  search 
areas  are  likely  to  be  used  only  as  a  back-up. 

System  reliability  is  an  issue  for  some  examiners,  despite  the  favorable  picture 
shown  by  survey  data  and  "hard"  data 

Overall  reliability  of  APS  and  its  supporting  networks  received  the  third  highest 
approval  rating  among  the  twenty-one  initial  factors  measured  in  the  survey.   This  was 
somewhat  surprising,  given  the  fact  that  poor  system  reliability  was  one  of  the 
complaints  from  many  examiners  interviewed.   As  one  examiner  said: 

You  run  into  a  system  outage,  a  freeze-up,  or  get  kicked  off  the  network 
for  no  apparent  reason  --  about  once  or  twice  a  month  --  and  several 
hours  of  work  may  go  down  the  tubes.12  That  happens  often  enough  to 
make  you  real  gun-shy  about  using  the  system  [especially  the  image 
workstation]. 


Earlier  versions  of  the  group  printer  were  somewhat  "fragile"  and  often  held  up  delivery  of 
documents  to  the  requesting  examiner.   Current  versions,  however,  are  much  improved.   The  availability 
standard  of  95%  or  higher  is  being  met,  with  rare  exception. 

System  outages  and  freeze-ups  are  particularly  frustrating  because  of  the  fact  that  some  of  PTO's 
automated  search  systems  -  particularly  the  shared  use  facilities  -  do  not  have  an  automatic  save  or 
back-up  feature.   If  an  examiner  has  worked  for  an  hour  on  a  search  and  the  system  goes  down,  that 
hour  of  work  is  lost  and  has  to  be  completely  reconstructed. 

59 


In  considering  the  issue  of  system  reliability,  the  following  points  should  be 
borne  in  mind: 

■  An  incident  of  "system  failure"  --  from  the  patent  examiner's  point-of-view 
--  could  be  the  result  of  a  mainframe  outage,  file  server  outage,  network 
outage,  software  failure  (application  specific),  workstation  failure,  or  even 
a  printer  failure.   Available  statistics  suggest  that  the  first  three  causes  -- 
mainframe  outage,  filer  server  outage,  or  network  outage  --  are  relatively 
rare  but  not  unknown.   But  add  these  together  and  factor  in  the 
additional  elements  of  "software  failure"  and  "local  equipment  failure" 
(reflected  to  some  extent  in  "Help  Desk  Incident  Reports"),  and  the  issue 
of  "system  reliability"  attains  significance  to  many  users. 

■  Regardless  of  the  precise  cause,  it  takes  only  one  or  two  instances  of 
losing  work  to  develop  a  jaundiced  view  of  "system  reliability."   Moreover, 
that  negative  perception  may  persist  several  months  after  a  problem 
situation  has  been  remedied.    It  is  possible  that  some  criticism  heard  in 
the  focus  groups  regarding  system  reliability  reflected  a  few 
"catastrophic"  failures  over  the  past  several  months. 

The  monthly  "Information  Systems  Executive  Report,"  produced  by  the  Capacity 
Planning  and  Performance  Management  Division,  provides  PTO's  senior  management 
with  the  best  available  "hard"  data  regarding  PTO's  "system  reliability."   One  section  of 
this  report  deals  with  "Messenger  [text]  Availability"  and  "CSIR  [image]  Availability"  -- 
which  reflects  primarily  maintenance  of  hardware  and  software.13  The  report  for 
August,  1994  shows  that  Messenger  availability  during  scheduled  operational  hours 
was  100%  that  month;  cumulative  availability  for  the  most  recent  twelve  month  period 
was  99.7%;  and  two  recent  "bad"  months  were  June,  1993  (slightly  higher  than  97% 
availability)  and  May,  1994  (slightly  lower  than  97%  availability). 

The  same  report  shows  that  CSIR  availability  was  100%  for  August,  1994; 
cumulative  availability  for  the  most  recent  twelve  month  period  was  99.7%;  and  two 
recent  "bad"  months  were  June,  1993  (98.5%  availability)  and  May,  1994  (97% 
availability). 

When  compared  to  both  government  and  private  sector  "mega-systems,"  the 
latter  figures  reflect  exemplary  performance.    However,  seeing  that  there  have  been 
months  when  text  or  image  was  "down"  even  3%  of  scheduled  hours  (and  total  down 
time  could  be  greater  than  that  in  certain  areas,  factoring  in  possible  localized 
problems  with  the  network),  it  is  understandable  that  at  least  a  few  examiners  might 
complain. 


13See  Appendix  D. 
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Another  section  of  the  same  report  --  titled  "Problem  Management  System 
Reports"  -  reflects  the  distribution  of  hardware  and  software  problems  reported 
through  the  Help  Desk  and  through  other,  more  direct  PTO  tracking  systems. 
Although  this  report  collapses  both  Messenger-related  and  CSIR-related  problems  into 
a  single  bar  labeled  "APS,"  it  appears  that  almost  300  "APS"  problems  were  observed 
in  August,  1994.   However,  the  report  also  reveals  the  towering  magnitude  of  PC- 
related  problems.   With  the  number  of  PTOnet  hook-ups  and  PC  usage  within  the 
examining  corps  undergoing  a  quantum  leap  during  this  period,  PC-related  APS 
problems  would  be  expected  to  go  up  proportionately. 

System  response  time  varies,  depending  on  the  APS  database  and  channel 
being  used 

Numerous  commentators  in  the  focus  group  and  interview  phase  reported  that 
APS  was  difficult  to  access  --  and  slow  to  respond  --  at  peak  periods  during  the  day 
and  at  certain  times  during  the  year.   Some  claimed  that  the  situation  deteriorated 
during  the  past  year  or  two  as  the  number  of  users  and  their  level  of  usage  has  gone 
up. 

Survey  findings  indicate  that  rating  of  "system  response  time"  varies 
considerably,  depending  on  whether  one  is  talking  about  image  retrieval  (CSIR)  or  text 
search  (Messenger);  whether  one  is  talking  about  "log  on/connect  time  to  initiate  a 
search,"  "time  to  execute  a  search  and  go  to  first  screen"  or  "time  to  go  screen-to- 
screen";  and  in  the  case  of  text  search,  whether  the  text  search  is  done  through  a 
desktop  PC,  through  a  shared,  single-screen  text  terminal,  or  through  an  image 
workstation.   These  ratings  are  summarized  in  the  following  "acceptance  scores" 
(percent  satisfied  or  neutral): 


Research  Tool 

Response  time  to 

execute  a  search 

and  go  to  first 

screen 

Response  time  to 
go  screen-to-screen 

Log  on/connect  time 
to  initiate  a  search 

Text  search  through  a 
desktop  PC 

71% 

81% 

81% 

Text  search  through  a 

single-screen,  shared 

text  terminal 

57% 

71% 

74% 

Text  search  through  an 
Image  workstation 

43% 

56% 

39% 

Image  retrieval  through 
an  Image  workstation 

40% 

49% 

40% 
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In  all  cases,  "response  time  to  go  screen-to-screen"  is  rated  at  least  somewhat 
more  favorably  than  "response  time  to  execute  a  search  and  go  to  first  screen." 
However,  in  the  case  of  image  retrieval  through  an  image  workstation,  less  than  half 
the  users  are  "satisfied"  or  "neutral"  on  any  measure;  log  on/connect  time  received  a 
particularly  abysmal  rating  of  thirty-nine  percent  (39%)  --  the  lowest  of  any  response 
measure.    In  the  case  of  text  search  through  a  desktop  PC,  approximately  three- 
quarters  of  all  users  are  "satisfied"  or  "neutral"  on  any  measure  --  this  is  the  top- 
ranked  channel  in  terms  of  system  response  time.   Text  search  through  a  single- 
screen,  shared  text  terminal  ranks  second  and  text  search  through  an  image 
workstation  ranks  third. 

The  best  "hard"  data  available  on  APS  response  time  is  conveyed  in  a  monthly 
"Information  Systems  Executive  Report"  produced  by  the  Capacity  Planning  and 
Performance  Management  Division.   A  "Transaction  Response  Time  for  Search 
Applications"  section  of  the  report  shows  the  monthly  average  response  time  for  both 
Messenger  search  requests  and  CSIR  page  retrievals.   Another  exhibit  titled 
"Messenger  Search  Requests  and  Response  Times"  shows  Messenger  response  time 
superimposed  on  the  number  of  Messenger  search  requests.14 

The  latter  time  series  graphs  show  that  the  monthly  number  of  Messenger 
requests  from  all  sources  -  examining  corps,  public  users,  and  other  --  has  trended 
upward,  from  about  150,000  in  October,  1991  to  about  233,000  in  August,  1994.    (An 
all-time  peak  of  262,566  search  requests  was  recorded  in  March,  1994.)   What  this 
means,  of  course,  is  that  the  actual  work  load  on  the  system  has  increased  over  sixty 
percent  (60%)  during  a  twenty-month  period  --  a  statistic  that  also  underscores  a 
dramatic  growth  in  dependency  on  APS. 

From  October,  1991  to  January,  1993,  Messenger  response  time  improved  -- 
from  24  seconds  to  about  16  seconds  --  largely  due  to  installation  of  Amdahl  6390-2 
storage  devices  in  May,  1992.    But  from  January,  1993  to  the  present  (August,  1994), 
Messenger  response  time  drifted  back  up  (deteriorated)  as  usage  continued  to 
increase.   As  of  August,  1994,  response  time  was  back  to  24.2  seconds  -  about  the 
same  level  as  it  was  at  in  October,  1991. 

Other  PTO  exhibits  state  that  PTO's  Messenger  response  time  target  is  for  "at 
least  80%  of  all  search  requests  to  be  completed  in  less  than  30  seconds."   In  August, 
1994,  69%  of  all  requests  were  completed  in  under  10  seconds  and  83%  were 
completed  in  less  than  30  seconds.    In  other  words,  Messenger  response  time  was 
within  target  by  only  a  small  margin  in  August,  1994. 


1  See  Appendix  D  for  details. 
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In  sum,  "hard"  data  indicate  that  Messenger  response  time  has  been  steadily 
deteriorating  in  recent  months,  and  this  situation  will  need  to  be  remedied  with 
appropriate  hardware  upgrades  in  the  near  future  to  avoid  placing  users  in  the  "critical 
zone." 

Examiners  also  raised  concerns  about  general  sluggishness  of  the  image 
workstation 

In  the  case  of  CSIR,  PNAs  (Page  Not  Available)  and  especially  unpredictable 
slowdowns  and  delays  (Request  in  Progress)  are  a  continuing  source  of  frustration  to 
the  corps  ("like  being  stalled  at  a  stoplight  for  five  minutes  or  more"). 

As  with  the  "stoplight"  metaphor,  occasional  delays  at  the  image  workstation 
seem  longer  than  they  really  are,  because  they  are  interrupting  a  work  sequence  and 
they  are  occurring  in  a  "dark  and  drafty  room,"  rather  than  at  "the  water  cooler."  While 
some  examiners  tend  to  focus  on  the  "time  lost"  in  using  the  image  workstation,  there 
is  a  tendency  to  overlook  the  time  gained  --  in  the  form  of  not  having  to  walk  to  a 
search  room  (sometimes  in  another  building),  pull  down  the  shoes,  sort  through  the 
paper,  and  put  the  whole  mess  back  in  order.   The  image  workstation  provides  further 
timesavings  by  enabling  the  user  to  create  document  collections  comprising  multiple 
class  and  subclass  by  automatically  merging  subclasses  and  eliminating  duplicates. 
This  enables  users  to  review  up  to  thirty  percent  fewer  documents  on  the  machine,  as 
compared  to  paper.15 

Internal  reports  indicate  that  the  total  number  of  image  pages  retrieved  (from  all 
sources  --  examiners,  public,  and  "other"  users)  has  gone  up  even  more  dramatically, 
from  less  than  four  million  in  January,  1993  to  over  twelve  million  in  August,  1994  --  a 
300%  increase  in  a  twenty-month  period.16   However,  at  the  same  time  that  aggregate 
demand  on  the  system  has  gone  up  to  that  extent,  average  retrieval  time  per  page, 
surprisingly,  has  actually  improved  --  from  about  .8  seconds  in  June,  1993  to  .44 
seconds  as  of  August,  1994. 


A  "more  recent  version"  of  the  image  workstation  also  has  the  welcome  advantage  of  being  much 
faster  to  log  on.   The  older  version  sometimes  required  five  minutes  or  more. 

16 

Much  of  that  300%  increase,  in  fact,  did  not  come  from  the  examining  corps,  but  from  the  fact 
that  a  Public  Image  Search  Facility  was  opened  in  July,  1993,  providing  public  access  to  CSIR  —  and  a 
spurt  to  total  CSIR  usage.   (As  of  September,  1994,  examiners  accounted  for  about  sixty  percent  (60%) 
of  all  image  pages  retrieved.)  The  point  to  be  made,  however,  is  that  response  time  experienced  by  the 
examiners  is  impacted  by  usage  from  all  sources;  total  usage  from  all  sources  is  shooting  up,  yet 
response  time  is  paradoxically  improving. 
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PTO  management  documents  further  state  that  PTO's  CSIR  target  is  "for  the 
retrieval  of  each  page  to  be  completed  in  less  than  one  second."  While  the  average 
retrieval  time  reached  a  peak  of  1.02  seconds  in  February,  1992,  performance  has 
been  well  ahead  of  target  since  that  month,  and  supposedly  has  been  getting  better 
up  to  the  present  time  (August,  1994).17 

"System  recoverability"  seems  to  vary  a  great  deal,  depending  on  the  APS 
channel  being  used 

Survey  findings  confirmed  that  system  recoverability  --  the  ability  to  save  work 
following  a  system  crash,  lock-up  or  boot-off  --  is  judged  best  with  text  search  through 
a  desktop  PC  (69%  acceptance);  next  best  with  text  search  through  a  shared,  single- 
screen  terminal  (63%  acceptance);  third  best  with  text  search  through  an  image 
workstation  (45%  acceptance);  and  worst  with  image  retrieval  through  an  image 
workstation  (44%  acceptance). 

This  accords  with  the  focus  group  finding  that  image  retrieval  through  an  image 
workstation  (at  least  in  its  recent  form)  is  particularly  lacking  in  automatic  backup-and- 
save  features,  while  text  search  through  a  desktop  PC  seems  to  provide  maximum 
safeguards  --  some  of  them  resident  in  the  desktop  PC  itself.18 


1  While  the  latter  "hard"  data  indicate  that  CSIR  retrieval  time  is  presently  excellent  —  and  has  been 
getting  better  throughout  the  past  three  years  -  focus  group  and  survey  results  strongly  suggest 
otherwise.    Pending  further  investigation,  we  suspect  the  "hard"  data  may  be  understating  the  actual 
delay  experienced  by  users  at  the  image  workstation  at  certain  times  and  in  certain  conditions.   This 
might  be  an  appropriate  topic  of  investigation  for  SIR,  including  comprehensive  on-site  measurement  of 
CSIR  retrieval  times  under  a  variety  of  times  and  conditions. 

This  is  only  one  of  several  instances  where  the  desktop  PC  --  particularly  the  operating  system 
and  the  graphical  user  interface  -  can  provide  flexibility,  convenience,  and  customization  features  that 
are  sometimes  lacking  in  the  more  standardized  "shared  facility"  channels. 
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PTOnet's  reliability  is  judged  to  be  acceptable  at  present  and  its  capacity 
adequate  to  handle  foreseeable  needs 

The  monthly  "Information  Systems  Executive  Report"  also  provides  PTO's 
senior  management  with  the  best  available  "hard"  data  regarding  PTOnet's  reliability. 
One  section  titled  "PTOnet  Outages"  shows  the  number  of  "segments,"  "applications," 
and  related  network  "devices"  down  during  that  month;  "possible  number  of  users 
affected";  and  "possible  user  hours  lost."  A  recent  report  for  August,  1994  shows  that 
there  were  25  instances  of  network  devices  down  during  that  month,  accounting  for 
1,725  possible  users  affected19  and  3,597.75  possible  user  hours  lost.20   (Some  of 
those  lost  hours  were  attributable  to  Trademark  and  other  divisions  of  PTO.) 

While  there  are  no  performance  targets  currently  in  effect  for  PTOnet  -  as  with 
Messenger  availability,  CSIR  availability,  and  workstation  availability  --  PTO  managers 
who  were  interviewed  about  PTOnet  performance  believe  the  figures  described  above 
are  quite  respectable  in  view  of  the  fact  that  there  are  101  network  segments  currently 
in  use  and  a  total  of  about  3,900  "customers"  on  the  network.   Their  reading  of  current 
Help  Desk  calls  is  that  user  problems  tend  to  be  related  to  the  platforms  and 
occasional  file  server  outages,  rather  than  to  network  availability  or  reliability. 

These  managers  also  believe  PTOnet's  reliability  has  been  improving  over  the 
past  year  or  two,  as  the  needs  of  various  divisions  and  usergroups  have  become 
better  understood  and  balanced  out  with  appropriate  network  configurations. 
However,  if  we  take  the  figures  described  above  at  face  value,  there  are  still  enough 
network  outages  each  month  to  "burn"  at  least  a  few  examiners  and  to  lose  a 
significant  amount  of  user-hours. 

On  October  21,  1994,  PRC  submitted  the  final  version  of  a  Systems  Analysis 
Report  (SAR)  on  PTO's  network  needs  through  the  year  2000.   The  report  projects  an 
increase  in  aggregate  throughput  from  26.59  Megabits  per  second  in  1994  to  352.82 
Megabits  per  second  in  2000,  and  discusses  in  detail  the  migration  plan  needed  to 
meet  future  demand.   This  report  appears  to  be  as  thorough  as  that  kind  of 
forecasting  document  can  be,  and  gives  reason  to  believe  that  PTO  can  stay  ahead  of 
the  curve  by  continuing  to  invest  in  its  network  in  a  steady  and  intelligent  manner  in 
coming  months. 


There  is  some  double  counting  in  this  figure. 

20 

These  data  are  included  in  Appendix  D. 
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In  sum,  well-informed  PTO  commentators  believe  that  PTOnet  is  not  a  "choke- 
point"  in  the  present  APS,  and  the  network  is  adequate  to  handle  "foreseeable"  needs 
in  the  near  future.   Nevertheless,  one  senses  an  element  of  unpredictability  as 
hundreds  of  examiners  are  scheduled  to  be  hooked  up  to  PTOnet  in  the  closing 
weeks  of  1994;  the  entire  examining  corps  converts  to  a  desktop  environment  (text 
and  image)  within  the  next  few  years;  and  many  new  office  automation  tools  come  on 
line.    PTO  must  pay  close  attention  to  emerging  demands  on  its  network  and  continue 
to  develop  that  infrastructure  in  a  steady  way. 
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Chapter  6:     TRAINING 


Examiners  lack  fundamental  skills  to  fully  utilize  the  information 
resources  available  to  them  at  PTO 

One  unmistakable  theme  emerging  from  the  focus  group  discussions  was  the 
need  for  better  training  about  how  to  use  the  myriad  of  APS-related  research  tools. 
Examiners  provided  anecdotal  examples  and  high  praise  for  the  original  training  they 
received  on  the  system,  but  almost  everyone  said  they  needed  on-going  and  refresher 
courses  to  fully  utilize  all  the  system  features  available  to  them.    Overall,  focus  group 
commentators  reported  a  pervasive  void  concerning  information  about  how  to  operate 
the  system  (documentation),  how  to  solve  common  problems  within  their  work  area, 
and  who  within  the  CIO  organization  had  responsibility  for  different  aspects  of  the 
system.    Focus  group  participants  often  got  sidetracked  trading  information  on  how  to 
"get"  APS  to  perform  some  function,  or  confessed  to  one  another,  "Gee,  I  never  knew 
the  system  could  do  that!" 

The  extent  to  which  PTO's  current  training  processes1  have  failed  to  meet  the 
most  basic  needs  of  examiners  is  borne  out  in  the  startling  finding  that  almost  half 
(46%)  of  the  examining  corps  are  dissatisfied  with  their  own  skills  at  using  available 
information  resources.   The  majority  of  examiners  received  initial  training  on  APS  Text 
Search,  yet  less  than  half  of  the  corps  can  actually  remember  and  utilize  most  of  what 
they  learned  in  the  initial  session.  And  although  examiners  rated  advanced  and 
refresher  training  as  one  of  PTO's  top  three  priorities,  only  a  small  fraction  of  the  corps 
has  ever  been  offered  such  training.   As  the  focus  group  commentators  revealed, 
most  examiners  have  only  a  basic  familiarity  with  APS  and  related  search  tools  and 
very  little  opportunity  to  learn  more  of  what  they  need  to  know  to  do  their  job 
effectively  and  efficiently. 


1PRC,  Inc.  was  responsible  for  Text  and  Image  training  in  the  period  July,  1990  to  July,  1993,  at 
which  time  the  PTO's  Patent  Academy  took  over  this  training.  According  to  the  APS  Text  and  Image 
Training  Metrics  prepared  by  PRC,  the  average  overall  scores  for  all  PRC-delivered  training  classes  fell 
in  the  "excellent"  category  (between  17  and  20  on  a  1-20  scale). 
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While  initial  training  may  be  adequate  (see  the  chart  on  the  previous  page), 
much  of  what  is  learned  is  lost  over  time.   Examiners  are  not  provided  with  regular 
opportunities  to  update  their  skills.    New  software  applications  become  essentially 
useless  because  examiners  often  do  not  know  that  they  are  available,  let  alone  how  to 
use  them.   Training  sessions  are  not  geared  towards  meeting  specific  art  group 
needs.   And  there  is  a  tremendous  lack  of  user-friendly  reference  materials  that 
explain  how  to  get  these  search  tools  to  do  what  examiners  need  them  to  do.    PTO 
needs  to  redirect  some  resources  into  this  apparently  weak  link  in  its  overall 
operational  strategy. 

Examiners  are  not  being  adequately  trained  on  APS  and  related  search  tools 

In  order  to  fully  document  the  level  of  training  at  PTO,  examiners  were  asked  a 
series  of  questions  relating  to  the  initial  and  refresher  or  update  training  they  received 
on  the  following:   search  strategies  and  methods  in  their  art  area;  text  search  on  APS; 
image  retrieval  on  an  image  workstation;  commercial  databases;  WinCom/WinNet;  or 
other  search  tools  that  have  been  made  available  to  them  at  PTO.   To  a  great  extent, 
their  responses  comported  with  our  focus  group  conclusions  -  while  examiners  are 
relatively  satisfied  with  the  initial  training  they  received  on  these  tools,  on-going 
refresher  and  update  training  is  sorely  needed  in  order  for  them  to  fully  utilize  all  of  the 
available  search  tools. 

Despite  the  generally  high  level  of  satisfaction  with  initial  training,  the  survey 
findings  reveal  that  its  value  is  diminished  because  a  majority  of  examiners  simply  do 
not  remember  what  they  learned  in  these  sessions.    More  importantly,  they  are  rarely 
provided  with  opportunities  to  update  or  advance  their  skills.    For  example,  most 
examiners  --  some  ninety-three  percent  (93%)  of  the  examining  corps  --  received  initial 
training  on  APS  Text  Search.   Of  these,  eighty-five  percent  (85%)  rated  that  training 
as  adequate  --  the  highest  score  received  by  any  training  program.   Yet  less  than  half 
(46%)  of  those  who  received  initial  training  on  Text  Search  remember  and  utilize  what 
they  were  taught  and  less  than  one-fifth  (12%)  have  received  refresher  or  update 
training  on  this  essential  search  tool. 

Examiners  reported  similar  training  experiences  with  other  search  tools.    Only 
twenty  percent  (20%)   --  or  ninety-one  percent  (91%)  of  the  twenty-two  (22%)  of  the 
total  examining  corps  who  currently  have  access  to  image  search  --  received  initial 
training  on  image  retrieval  on  an  image  workstation.   Of  that  twenty  percent  (20%), 
almost  three-quarters  (72%)  rated  the  initial  training  as  adequate  and  just  over  half 
(55%)  still  remember  what  they  learned.   A  paltry  two  percent  (2%)  of  examiners  with 
access  to  image  search  have  received  refresher  or  update  training.   These  results 
indicate  that  there  are  some  2%  of  examiners  who  currently  have  access  to  image 
search  that  have  never  received  training  on  this  critical  search  tool. 
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One  of  the  most  dramatic  findings  is  that  some  three-fifths  (59%)  of  the 
examining  corps  has  never  received  initial  training  on  search  strategies  and  methods 
for  their  specific  art  area.   This  is  a  particularly  disturbing  finding  given  the  fact  that 
examiners  have  strikingly  different  needs  for  electronic  tools.   The  APS  Text  and 
Image  Search  systems  have  been  developed  as  generic  search  tools  and  examiners 
have,  for  the  most  part,  been  provided  with  generic  training  sessions.    Examiners  do 
not,  however,  have  generic  searching  techniques.  The  154  art  groups  --  the 
subgroups  of  PTO's  16  chemical,  electrical  and  mechanical  examining  groups  --  have 
very  different  needs,  in  terms  of  search  strategies,  work  methods  and  the  search  tools 
that  must  be  applied  in  the  examination  of  each  patent  application. 

The  survey  revealed  that  examiners  are  very  dissatisfied  with  the  -extent  to 
which  APS  is  customized  to  art  group  needs.   This  measurment  received  the  fourth 
lowest  score  in  a  section  which  rated  satisfaction  and  importance  of  various  aspects  of 
PTO's  information  resources.   Moreover,  two-thirds  (65%)  of  the  examining  corps  said 
that  advanced/refresher  training  on  new  APS  features,  "tailored  to  the  way  search  is 
conducted  in  each  of  their  specific  art  areas"  would  have  a  significant  impact  on  their 
job  performance.   This  latter  item  was  rated  as  the  second  top  priority  in  a  list  of  fifty- 
five  potential  enhancements  to  the  Automated  Patent  System. 

Of  the  two-fifths  (41%)  of  examiners  who  did  receive  initial  training  on  art  group 
specific  search  strategies  and  methods,  less  than  three  quarters  (71%),  rated  the 
session  as  adequate.   Less  than  half  (46%),  still  remember  some  or  most  of  what  they 
learned  in  that  initial  session.   Refresher  or  update  training  on  art-specific  search 
strategies  and  methods  was  provided  to  only  seven  percent  (7%)  of  the  corps,  of 
which  sixty-two  percent  (62%)  rated  this  session  as  adequate. 

As  with  image  retrieval,  only  a  small  subset  of  the  total  examining  corps  is 
currently  able  to  access  the  commercial  databases  which  are  made  available  to 
examiners  in  the  following  locations:   through  the  shared  use  text  terminals  and  the 
image  workstations;  at  desktop  PCs;  at  special  terminals  throughout  the  offices  which 
house  examiners;  and,  at  the  Scientific  and  Technical  Information  Center  (STIC). 
According  to  this  study,  less  than  half  (46%)  of  the  total  examining  corps  reports  they 
have  access  to  one  or  more  commercial  databases.   Yet  just  over  half  (51%)  indicate 
they  received  initial  training  on  one  or  more  commercial  databases.   This  discrepancy 
could  be  explained  by  the  fact  that  examiners  must  have  an  account  number  to 
access  a  commercial  database.    It  is  possible  that  some  examiners  were  never  given 
an  account  number  to  actually  use  the  commercial  database  for  which  they  received 
training. 
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Next  to  APS  Text  Search,  initial  training  on  commercial  databases  received  the 
second  highest  score  for  training  adequacy  --  seventy-two  percent  (72%)  who  received 
it  were  satisfied  with  their  initial  training  session.   Commercial  databases  also  received 
the  highest  scores  in  the  refresher/update  training  category.   Of  the  one-fifth  (21%)  of 
examiners  who  received  refresher/update  training  on  commercial  databases,  over 
three-quarters  (76%)  were  satisfied  with  that  training.   On  the  other  hand,  commercial 
databases  received  the  lowest  score  --  twenty-four  percent  (24%)  --  for  the  number  of 
examiners  who  remember  what  they  learned  in  their  initial  training  session. 

Although  almost  the  entire  examining  corps  (93%)  has  a  PC  at  their  desk,  just 
over  one-tenth  (14%)  are  currently  using  WinCom.   Forty-six  percent  (46%)  use 
WinNet,  sixteen  (16%)  don't  use  either,  and  almost  one-quarter  of  the  corps  (24%), 
are  not  sure  if  they  are  using  WinCom  or  WinNet  on  their  desktop  PC.    In  addition, 
only  one-fifth  (20%)  of  the  examining  corps  has  received  initial  training  on  either 
WinCom  or  WinNet.    Less  than  two-thirds  (62%)  --  the  lowest  score  in  this  category  -- 
who  received  this  initial  training  rated  it  as  adequate.   Surprisingly  only  forty-five 
percent  (45%)  remember  what  they  learned,  despite  the  fact  that  this  training  would 
have  taken  place  within  the  last  year.   Not  surprisingly,  only  three  percent  (3%)  have 
received  refresher/update  training  on  WinCom/WinNet;  only  two-fifths  (40%)  of  those 
who  did  receive  it,  rated  it  as  adequate. 

A  final  search  tool  category,  "other,"  yielded  some  interesting  results.   Some 
one-fifth  (19%)  of  the  examining  corps  received  initial  training  on  other  automated 
tools  and  applications,  such  as  the  Internet,  Action  Writer  and  PCs.   Three-quarters 
(73%)  of  those  who  received  initial  training  on  these  other  applications  rated  this 
training  as  adequate;  yet,  only  two-fifths  (40%)  still  remember  most  of  what  they 
learned  during  this  session.   Of  the  three  percent  (3%)  of  examiners  who  received 
refresher  or  update  training  on  these  other  search  tools,  some  three-fifths  (61%)  were 
satisfied  with  this  session. 

Examiners  were  asked  to  rate  the  adequacy  of  documentation  and  reference 
materials  they  have  received  in  each  of  the  following  areas:   search  strategies  and 
methods  in  their  art  area;  text  search  on  APS;  image  retrieval  on  an  image 
workstation;  commercial  databases;  WinCom/WinNet;  or  other  search  tools  that  have 
been  made  available  to  them  at  PTO.   Focus  group  commentators  reported  a 
pervasive  void  concerning  documentation  about  how  to  operate  APS  and  supporting 
systems.   The  APS  manual  was  declared  impenetrable,  the  irregular  APS  tips  in  the 
PTO  Newsletter  hard  to  follow,  and  the  overall  documentation  on  new  features  and 
enhancements  virtually  non-existent. 
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Statistical  findings  support  the  focus  group  observations  about  documentation  -- 
one-third  to  two-thirds  of  the  examining  corps  find  their  documentation  and  reference 
materials  to  be  inadequate,  depending  on  the  area  in  question.    Examiners  are  most 
satisfied  with  documentation  on  APS  Text  Search:   over  two-thirds  (68%)  of  the  corps 
rating  it  as  very  or  generally  adequate.   Commercial  databases  rated  second  at  sixty- 
one  percent  (61%)  and  other  automated  tools  rated  fifty-nine  percent  (59%). 
Approximately  fifty-five  percent  (55%)  of  the  examining  corps  is  satisfied  with 
documentation  and  reference  materials  received  for  image  retrieval  on  the  image 
workstations.   At  the  bottom  of  the  documentation  satisfaction  rung  are  search 
strategies  and  methods  for  art  areas  and  WinCom/WinNet,  both  at  forty-two  percent 
(42%). 

Overall,  patent  examiners  express  serious  misgivings  about  their  ability  to  fully 
utilize  the  information  resources  available  to  them  at  PTO.   When  asked  to  rate  the 
extent  to  which  they  are  satisfied  that  they  have  the  training  and  skills  to  effectively 
and  efficiently  use  all  the  available  information  resources,  about  half  (46%)  of 
examiners  stated  that  they  are  dissatisfied.    Examiners  clearly  want  and  need  more 
and  better  training  on  all  of  the  automated  systems  and  tools  that  they  use  to  do  their 
job. 

Examiners  are  most  satisfied  with  APS  Text  Search  training  and  documentation 

Of  all  of  the  search  tools  available  to  examiners,  PTO  has  done  the  best  job  of 
preparing  the  corps  to  access  and  manipulate  APS  Text  Search.    Examiners  gave 
Text  Search  on  APS  the  highest  score  in  the  following  categories: 

■  Percentage  of  the  corps  who  received  initial  training  (93%) 

■  Percentage  of  the  corps  who  rated  the  initial  training  as  adequate  (85%) 

■  Percentage  of  the  corps  who  rated  documentation  as  adequate  (68%) 

Text  Search  on  APS  rated  the  second  highest  in  two  other  categories: 

■  Percentage  of  the  corps  who  received  refresher/update  training  (12%) 

■  Percentage  of  the  corps  who  rated  the  refresher  training  as  adequate  (65%) 

These  positive  ratings  for  APS  Text  Search  are  probably  related  to  the  fact  that 
all  examiners  have  access  to  it  -  either  from  a  shared  text  terminal,  an  image 
workstation  or  from  a  desktop  PC  --  and  most  examiners  use  it  on  at  least  some 
applications. 
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Regular  use  of  search  tools  reinforces  training 

The  focus  group  findings  and  statistical  data  demonstrate  the  linkage  between 
the  extent  to  which  examiners  remember  what  they  learned  during  their  initial  training 
and  the  extent  to  which  they  use  the  various  search  tools  on  a  regular  basis.   While 
only  a  small  percentage  of  the  corps  --  twenty-two  percent  (22%)  --  currently  has 
access  to  image  retrieval,  these  examiners  have  the  highest  recall  rating  from  their 
initial  training  --  over  half  (55%)  remember  most  of  what  they  learned.    Our  findings 
show  that  examiners  with  image  retrieval  access  tend  to  be  committed  to  this  system 
and  to  use  it  on  a  fairly  regular  basis.   Sixty-three  percent  (63%)  of  them  say  that  they 
use  it  nearly  every  day  or  at  least  one  or  more  times  a  week.   Over  three-quarters 
(78%)  feel  that  image  retrieval  largely  fulfills  the  information  needs  of  their  job  -  the 
second  highest  rating  for  all  of  PTO's  automated  search  tools. 

At  the  other  extreme,  of  the  fifty-one  percent  (51%)  of  the  examining  corps  who 
were  trained  to  use  one  or  more  commercial  databases,  only  one-quarter  (24%) 
remember  what  they  learned  in  their  initial  training  session.   While  the  need  is  growing 
for  examiners  to  search  the  non-patent  technical  literature  available  through 
commercial  databases,  there  is  currently  only  a  small  percentage  of  examiners  who 
use  this  search  tool  on  a  regular  basis.  Over  half  (54%)  of  the  corps  do  not  currently 
have  access  to  any  commercial  databases.   Of  those  who  do  have  access,  only  seven 
percent  (7%)  use  commercial  databases  at  a  special  center  or  a  shared  facility 
workspace  on  a  regular  basis.   Sixteen  percent  (16%)  use  commercial  databases 
through  their  desktop  PC  on  a  regular  basis. 

The  survey  data  clearly  shows  that  examiners  want  more  and  better  access  to 
commercial  databases.   Almost  three-fifths  (59%)  of  those  examiners  who  do  not 
currently  have  access  to  commercial  databases  would  like  to  be  able  to  use  them,  and 
over  one-third  (36%)  of  these  examiners  think  they  would  use  these  search  tools  at 
least  one  or  more  times  per  week.    Moreover,  the  ability  to  access  all  commercial 
databases  at  the  desktop  was  rated  as  one  of  PTO's  top  fifteen  priorities  in  a  list  of 
fifty-five  potential  system  enhancements.    PTO  must  find  a  way  to  better  prepare 
examiners  to  make  full  use  of  these  increasingly  vital  search  tools. 
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Refresher/update  training  is  inadequate  by  any  standard 

Examiners  across  the  board  are  extremely  dissatisfied  with  the  state  of 
refresher  or  update  training  they  received  --  or  failed  to  receive  --  at  PTO.    Focus 
group  commentators  appeared  dumbfounded  when  asked  what  they  thought  about 
their  refresher  and  update  training  sessions  and  the  study  results  illustrate  why.   Very 
few  of  them  have  received  any  training,  of  any  kind,  beyond  their  initial  training.   Only 
twelve  percent  (12%)  of  the  examining  corps  has  participated  in  refresher/update 
training  for  Text  Search  on  APS.   An  even  smaller  percentage  (7%)  has  participated  in 
refresher/update  training  for  search  strategies  and  methods  for  their  art  area.   Only  a 
handful  (3%)  of  the  corps  has  received  refresher/update  training  for  WinCom/WinNet. 
Of  those  examiners  who  have  access  to  image  retrieval  --  arguably  the  most  complex 
of  APS's  subsystems  --  only  two  percent  (2%)  have  participated  in  refresher/update 
training  for  image  retrieval  on  an  image  workstation.   And  only  one  fifth  (21%)  of  those 
who  have  access  to  a  commercial  database  have  participated  in  refresher/update 
training  for  this  search  tool. 

Moreover,  refresher/update  training,  infrequent  and  tardy  as  it  is,  is  significantly 
inadequate  when  compared  to  initial  training.   For  example,  under  two-thirds  (65%)  of 
the  examiners  who  participated  in  refresher  training  on  APS  Text  Search  rated  this 
session  as  adequate,  as  compared  to  eighty-five  percent  (85%)  who  rated  their  initial 
training  as  adequate.   Over  three-fifths  (62%)  rated  update  training  for  art-specific 
search  strategies  and  methods  as  adequate;  just  under  three-quarters  (71%)  rated 
their  initial  training  for  this  search  tool  as  adequate.   WinCom/WinNet  received  the 
lowest  rating  in  the  satisfaction  with  update  training  category  --  two-fifths  (40%)  rated  it 
as  adequate.   Over  three-fifths  (62%)  rated  their  WinCom/WinNet  initial  training  as 
adequate.   Only  one  search  tool  --  commercial  databases  -  received  a  higher  rating 
for  update  training  (76%)  over  initial  training  (72%). 
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Chapter  7:     COMMUNICATION,  MANAGEMENT  AND 
SUPPORT  ISSUES 


Communication  channels  between  examiners  and  the  managers  and 
divisions  of  PTO  which  support  APS  impact  success 

Commentators  in  the  focus  group  sessions  described  an  environment  at  PTO  in 
which  effective  channels  of  communication  are  either  non-existent  or  practically 
useless.    Examiners  do  not  know  where  to  address  their  complaints  or  suggestions  for 
improvements  within  PTO  management  and  most  tended  to  be  pessimistic  in  their 
view  that  nothing  would  ever  happen  even  if  they  were  to  make  constructive 
suggestions.   There  are  few  opportunities  for  examiners  to  exchange  information, 
whether  it  be  technical  developments  in  a  given  art  or  tips  on  how  to  get  APS  to  do 
something  or  apply  advanced  search  techniques.   Written  documentation  on  system 
changes  and  enhancements  is  not  only  untimely,  but  also  poorly  delivered  and 
organized.   The  survey  findings  confirm  conclusions  drawn  from  the  focus  group 
sessions  that  PTO  must  make  improved  communication  one  of  its  top  priorities. 

Past  communication  channels  have  not  been  effective 

Examiners  who  participated  in  the  focus  group  sessions  were  virtually 
unanimous  in  their  belief  that  there  are  currently  few  effective  channels  for 
communicating  suggestions  for  systems  related  improvements  to  the  managers  who 
are  making  the  important  decisions  about  what  APS  will  look  like  in  five  years.    Few 
examiners  are  aware  of  the  user  group  representative  concept  and  those  who  do 
doubt  that  these  representatives  can  really  affect  change.   Moreover,  examiners  tend 
to  feel  that  even  when  they  do  make  suggestions,  they  never  receive  feedback  on 
their  input. 
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Some  of  the  more  senior  examiners  who  participated  in  the  focus  groups 
opined  that  this  lack  of  communication  was  the  result  of  deliberate  management 
philosophy  that  caused  system  development  and  management  to  be  partitioned  off 
from  the  view  of  most  managers  as  well  as  of  front-line  employees.    During  the 
handful  of  group  meetings  with  middle  managers,  someone  almost  invariably  (and 
quite  spontaneously)  claimed  that  one  recent  regime  of  information  resource 
managers  had  applied  authoritarian  approaches  to  system  development  and 
intentionally  excluded  other  organizations  from  influencing  decision-making  during  a 
several  year  long  epoch.1 

When  asked  to  rate  their  satisfaction  with  and  the  importance  of  some  twenty- 
one  aspects  of  APS,  examiners  indicated  that  one  of  the  greatest  areas  of 
disappointment  is  communication  --  or  the  lack  there  of.   Along  with  refresher/update 
training  (51%  satisfied),  the  availability  of  a  channel  for  submitting  systems-related 
suggestions  for  improvements  to  APS  rated  the  lowest  in  terms  of  satisfaction  (53% 
satisfied).   Just  over  half  (56%)  are  satisfied  with  the  feedback  they  receive  on 
suggestions  they  have  submitted  about  problems  regarding  APS. 

Examiners  are  also  not  satisfied  with  PTO's  track  record  in  preparing  examiners 
for  system  changes  and  enhancements  --  both  in  terms  of  training  and  documentation. 
Focus  group  commentators  complained  of  system  changes  being  made  to  their 
desktop  PC  without  any  warning  or  the  provision  of  related  documentation.    It  was  not 
surprising  to  find  that  examiners  are  not  satisfied  with  the  reliability  of  information 
about  system  changes  and  enhancements  scheduled  for  implementation  (only  64% 
were  satisfied). 

Two-thirds  (66%)  of  the  examining  corps  are  satisfied  with  the  opportunities 
they  have  for  swapping  information  among  colleagues  about  search  methods  and 
techniques. 


1A  number  of  re-alignments  and  reorganizations  of  the  information  resource  management 
organization  and  operations  has  occurred  in  the  past  year.  These  changes  are  believed  by  many  well- 
informed  commentators  to  reflect  needed  reforms  to  introduce  better  communication  and  cooperation. 
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Communication  about  needed  system  requirements  travels  along  a  convoluted 
path  between  examiners  and  those  who  bear  ultimate  responsibility  for 
responding  to  their  needs. 

The  voices  of  the  various  examining  groups  are  transmitted  through  a  series  of 
organizational  trumpets  with  various  roles  in  telling  the  examiner's  story  --  including 
individual  group  leaders,  their  union  leadership,  the  SIR  organization  and  the  CIO 
community  --  before  their  requirements  are  conveyed  finally  to  senior  management 
and  on  to  the  vendor(s).   Along  this  convoluted  path,  many  translators  intervene  -- 
often  with  conflicting  interests  --  before  matters  can  be  negotiated  and  resolved. 

Before  any  final  task  order  can  be  handed  off  to  vendors,  a  certain  erosion  of 
clarity  is  almost  inevitable  in  the  current  management  process.   One  useful 
organizational  model  that  has  been  employed  successfully  in  the  private  sector 
assembles  customer-oriented  teams  to  focus  on  meeting  end-user  evolving  needs. 
Such  teams  are  constantly  reviewing  ways  to  adopt  the  best  technology  to  address 
emerging  business  requirements,  re-defining  applications  as  technology  shifts  and 
providing  all  the  technical  support  a  customer  needs  to  utilize  the  very  best  technology 
available  at  the  best  price.   While  there  are  some  aspects  of  these  strategies  currently 
available  in  the  function  of  SIR,  more  direct  communication  between  the  primary 
system  users  and  the  system  developers  could  go  a  long  way  to  alleviate  some  user 
resistance  and  more  efficient  use  of  APS. 

Anecdotal  and  statistical  data  present  conflicting  information  about 
the  Help  Desk 

Focus  group  commentators  were  quick  to  claim  that  the  Government  Client 
Support  Help  Line  (the  "Help  Desk")  is  not  very  helpful.   They  attributed  the  situation 
to  a  number  of  factors:   personnel  who  staff  the  Help  Desk  are  not  as  "technically 
astute"  as  they  should  be;  they  are  expected  to  be  experts  on  a  variety  of  systems  in 
addition  to  APS;  they  do  not  understand  the  larger  automation  context  within  which 
APS  works;  and  they  have  to  hand  off  certain  problems  to  ClO's  Technical  Control 
Facility  (TCF),  where  additional  delays  in  problem  resolution  may  occur. 
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However,  when  examiners  were  asked  to  rate  their  satisfaction  on  a  list  of 
twenty-one  IRM-related  issues,  fully  eighty-two  percent  (82%)  of  all  respondents 
described  themselves  as  "satisfied"  or  "neutral"  on  the  issue  of  the  "effectiveness  of 
the  Help  Desk  (305-9000)  in  resolving  equipment  problems  or  system  problems  (they) 
encounter  in  the  course  of  doing  (their)  job."   Less  than  one-fifth  (18%)  were 
dissatisfied  --  ranking  this  item  near  the  top  of  all  twenty-one  items.2 

On  the  other  hand,  the  survey  reveals  strong  dissatisfaction  concerning 
"availability  of  someone  who  can  help  get  you  through  some  of  the  more  difficult  or 
obscure  situations  that  sometimes  arise  with  APS  (only  64%  were  satisfied  or  neutral; 
36%  were  dissatisfied).   Apparently  patent  examiners  do  not  make  the  connection  that 
the  Help  Desk  might  be  expected  to  perform  the  latter  function. 

The  Information  Systems  Executive  Report  (September,  1994),  particularly  the 
page  titled  "Problem  Management  System  report,"  sheds  some  light  on  the  Help  Desk 
performance: 

■  The  total  number  of  APS  problems  in  September,  1994  was  about  300. 

■  Approximately  90%  of  those  300  APS  problems  were  resolved  within  0-3 
days;  the  average  resolution  time  was  five  days. 

■  The  "PC"  category  towers  over  all  other  categories,  with  over  800 
problem  records  noted.  This  is  partly  explainable  by  the  fact  that  the 
"PC"  category  is  independent  of  any  application  category,  and  includes 
such  items  as  requests  for  PC  software  upgrades,  installations,  and 
moves.   With  PTO's  impending  shift  to  a  desktop  environment,  this  large 
category  is  certain  to  become  even  larger. 

■  A  PC  Support  Help  Desk  (308-6936),  provided  by  contractor  GMSI, 
handles  all  PC-related  problems,  questions  and  requests  from  all  areas 
of  PTO.   Some  six  persons  respond  to  an  average  call  volume  of  fifty 
calls  per  day  (as  of  early  November,  1994). 


2While  not  wishing  to  detract  from  this  positive  and  somewhat  surprising  result,  we  should  point  out 
that  this  question  was  asked  of  all  examiners,  with  no  further  qualification  regarding  the  respondent's 
amount  of  experience  (or  recency  of  experience)  with  the  Help  Desk.   (We  will,  however,  be  able  to  see 
if  this  result  holds  up  as  well  with  respondents  who  identify  themselves  as  relatively  heavy  users  of 
APS.) 

Until  we  have  more  in-depth  analysis  on  the  issue,  it  appears  that  the  Help  Desk  is  not  a  major  source 
of  dissatisfaction  to  the  examining  corps  as  a  whole. 
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■         The  regular  USPTO  Help  Desk  (305-9000)--  staffed  by  about  thirteen 
PTO  employees  --  handles  problems  related  to  specific  applications, 
such  as  APS  devices  --  workstations,  group  printers,  etc  --  and  the 
respective  platforms.   Call  volume  to  this  group  is  currently  averaging 
about  forty  calls  per  day. 

PTO  management  envisions  a  Help  Desk  configuration  for  PTO  that  would  be 
mostly  decentralized,  with  dedicated,  on-site  personnel  assigned  to  at  least  four 
locations:   Office  of  Trademarks,  Office  of  Patents,  Associate  Commissioner,  and 
OCTO/CIO.   Support  on  weekends  and  holidays  might  be  furnished  from  a  single, 
centralized  location. 

As  this  study  approached  its  conclusion,  a  dedicated  on-site  Help  Desk  had 
been  recently  installed  at  the  Office  of  Trademarks,  and  plans  were  underway  to 
introduce  a  similar  arrangement  for  Patents.   Such  a  decentralized  arrangement 
should  lead  to  improved  service  to  the  user  groups,  and  improved  feedback  from  the 
user  groups  to  the  people  responsible  for  providing  production  facilities. 
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Chapter  8:     FUTURE  NEEDS  ASSESSMENT 


APS  is  a  living  system  and  both  expectations  and  requirements  shift 
as  new  technology  emerges 

During  the  interview  phase,  many  commentators  seemed  caught  in  a  dilemma. 
On  the  one  hand,  they  praised  APS  as  the  best  of  all  possible  worlds,  but  often  in  the 
next  breath  they  cursed  it  because  it  could  not  help  them  accomplish  even  more. 
Overall,  examiners  have  come  to  take  APS's  features  for  granted  and  in  the  process 
have  continued  to  heighten  their  expectations  about  what  the  system  can  and  cannot 
do.   One  examiner  put  it  this  way: 

If  I  had  to  do  without  APS,  it  would  be  like  asking  me  to  give  up 
electricity  when  my  complaints  are  only  that  I  need  a  halogen 
lamp  instead  of  a  60°  watt  bulb.    If  we  complain,  it  is  because  we 
need  more  --  not  less. 

During  one  discussion  about  whether  APS  was  able  to  stay  abreast  of  current 
technology  and  remain  responsive  to  examiners'  emerging  needs,  another 
commentator  made  this  observation: 

"We  ought  not  pillory  the  pioneers  for  trying.   When  we  started  out 
ten  years  ago,  no  one  had  ever  made  this  journey.    PTO  was 
trying  to  do  something  that  had  never  been  done  before.   The 
technology  was  back  at  the  abacus  stage  and  PTO  forged  ahead, 
based  on  the  best  knowledge  of  what  was  available  at  the  time. 
But  now  we  are  behind  and  need  to  accelerate  in  order  to  get  out 
ahead  of  the  curve  again.   This  is  just  part  of  the  evolutionary 
cycle." 
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Another  examiner  --  among  the  few  who  claimed  to  be  a  real  computer  expert 
--  made  this  comment: 

"By  the  standards  of  the  mid-1990s,  APS  is  a  very  cumbersome, 
unforgiving,  sharp-edged,  hard-to-leam,  hard-to-remember,  non-user- 
friendly  system.   When  I  think  about  the  overall  structure  of  commands  in 
APS  and  its  Boolean  logic  terms  for  specifying  search  parameters,  the 
word  that  keeps  coming  to  mind  is  'arcane.'   But  this  is  a  hills  and  valleys 
business  and  we  will  probably  always  be  either  behind  or  ahead.    In  the 
meanwhile  --  probably  most  of  the  time  --  we  are  in  the  mainstream  of 
development  and  we  keep  relying  on  APS  to  do  the  business  of  both 
yesterday  and  tomorrow." 

APS  flexibility  to  respond  to  individual  group  needs  should  be  a  priority 

There  is  no  question  that  one  disappointing  --  although  perhaps  unavoidable  -- 
outcome  of  APS's  original  architectural  strategy  is  its  relative  inflexibility  in  responding 
to  the  full  range  of  often  unique  requirements  within  individual  art  units.   As  one 
examiner  put  it,  "APS  is  a  generic  searching  tool"  and  "examiners  don't  have  generic 
searching  techniques."   Other  commentators  on  this  subject  had  this  to  say: 

"APS  was  designed  to  mimic  what  we  thought  to  be  the  standard  mode 
of  paper  search  then  in  effect  throughout  the  examining  corps.    In  fact, 
that  'standard  mode'  turned  out  to  have  more  local  variations  than 
anyone  realized." 

"APS  is  a  one-size-fits-all  kimono.   There  is  much  need  to  customize  it  -- 
in  a  more  user-friendly,  menu-driven  way  --  to  the  unique  needs  and 
methods  of  each  art  area." 

The  survey  results  indicate  that  nearly  half  (43%)  are  dissatisfied  with  the 
extent  to  which  APS  has  been  customized  to  meet  the  individual  needs  of  the 
examining  groups. 

Focus  group  discussions  provided  the  basis  for  creating  the  list  of  most 
requested  features 

Each  examining  group  to  participate  in  the  interviews  generated  its  own  "wish 
list"  of  feature  enhancements  they  wanted  or  needed.    While  some  had  idiosyncratic 
application  to  only  one  or  two  sub-groups,  most  requests  traversed  the  boundaries  of 
examining  and  art  groups.    By  the  time  the  interview  phase  had  ended,  a  basic  list  of 
fifty-five  items  emerged  as  the  features  most  requested  across  the  examining 
community. 
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The  examiners'  "wish  list"  was  transformed  into  the  Future  Needs  Assessment 
section  of  the  survey.   Respondents  were  asked  to  indicate  what  impact  they  believed 
a  given  feature  would  have  on  their  overall  productivity  it  were  available,  using  the 
following  scale: 

Major  Impact 
High  Impact 
Some  Impact 
Little  Impact 

In  analyzing  and  reporting  data  from  this  section,  the  top  two  "high  impact" 
categories  were  bundled,  as  were  the  lower  two  having  lesser  impact. 

The  table  which  follows  reports  --  in  rank  descending  order  of  perceived 
positive  impact  --  the  fifty-five  future  features  rated  by  examiners.   The  table  also 
reports  the  current  status  of  PTO  efforts  to  respond  to  these  needs. 

One  cautionary  note  is  in  order.  The  table  reports  the  priorities  of  the  entire 
examining  corps.  Because  of  the  unique  requirements  of  the  examining  groups,  the 
rank  descending  order  is  different  with  each  separate  community.  For  instance,  the 
"ability  to  search  on  a  chemical  structure"  is  near  the  bottom  of  the  generic  list  -  51 
out  of  55.  Yet,  over  sixty-one  percent  (61%)  of  chemical  group  respondents  believe 
that  such  a  feature  will  have  major  impact  on  their  productivity.  Follow-on  analyses  of 
each  group  will  be  needed  to  establish  group-specific  priorities. 
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Chapter  9:    DEMOGRAPHICS 


Responses  to  the  survey  were  evenly  distributed  across  the  entire  examining 
group  as  the  chart  below  illustrates: 


Electrical 


Mechanical 


33% 


w^| 

HHPft 

-32% 

Chemical 

Examining  Group  Type 

Likewise,  a  good  representation  came  from  the  different  examining  groups, 
allowing  follow-up  analyses  that  can  associate  specific  problem  areas  with  specific 
examining  groups. 


Name  ol  Examining  Group 

\Q49.   Which  examining 
group  are  you  currently 
{assigned  to? 

|    %  Response 
distribution 

Number  ol 
Respondents 

58 

General,  Metallurgical,  Inorganic,  Petroleum  and 
Electrical  Chemical  and  Engineering 

1100 

8.00 

Organic  Chemistry  Drug,  Bio-Aflecting  and  Body  Treating 
Composition 

1200 

4.00 

26 

Specialized  Chemical  industries  and  Chemical 
Engineering 

1300 

500 

34 

High  Polymer  Chemistry.  Plastics.  Coating.  Photography. 

1500 

600 

Stock  Materials  and  Composition 

Biotechnology 

1800 

900 

64 

Industrial  Electronics 

2100 

6.00 

40 

Special  Laws  Administration  &  Designs  •  Utility  Patents 

2200 

300 

18 
51 

Information  Processing,  Storage  and  Retrieval 

2300 

700 

Packages.  Cleaning,  Textiles  and  Geometrical 
Instruments 

2400 

500 

32 

Electronic  and  Optical  Systems  and  Devices 

2500 

11.00 

77 

Communications.  Measuring.  Testing  and 
Lamp/Discharge 

2600 

10  00 

69 

Special  Laws  Administration  &  Designs  -  Designs  Patents 

2900 

200 

16 

Handling  and  Transportation  Media 

3100 

4.00 

31 
32 

Material  Shipping.  Article  Manufacturing.  Tools 

3200 

5.00 

Surgery,  Animal  Husbandry,  Medical  Technology, 

3300 

600 

40 

Amusements  and  Exercise  Devices  and  Printing 

Solar  Heat,  Power  and  Fluid  Engineering  Devices 

3400 

5.00 

35 

General  Construction,  Petroleum  and  Mining  Engineering 

3500 

6.00 

40 
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The  examining  corps  is  comprised  of  highly-educated  professionals  who  are,  for 
the  most  part,  well  along  on  the  federal  career  path.   While  all  examiners  have  --  at 
the  least  --  earned  a  baccalaureate  degree,  sixteen  percent  (16%)  are  law  school 
graduates  and  forty-eight  percent  (48%)  have  a  graduate  or  technical  degree  in 
another  discipline.   Another  thirty-six  percent  (36%)  have  completed  work  towards  a 
graduate  degree.   These  educational  attainments  are  reflected  in  this  chart: 


Some  Graduate  School 


Law  School 


Graduate/Professional 


Higher  Education  Attainment 
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Respondents  were  about  equally  divided  between  those  who  had  more  or  less 
than  five  years  of  experience  as  a  federal  employee  --  although  nearly  one-third  (31%) 
had  more  than  ten  years  of  service.   The  actual  employee  service  distribution  was: 


3-5  Years 


1-3  Years 


5-10  Years 


Under  1  Year 


Over  10  Years 


Years  of  Federal  Service 
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Nearly  seven  out  of  ten  (69%)  of  all  respondents  have  federal  pay 
classifications  of  above  the  GS-13  level.    One-fourth  (25%)  hold  journeyman  GS-9  to 
12  classifications.   A  smaller  group  (7%)  are  entry  level  examiners  --  GS-5  to  8 
classification.    Another  smaller  group  (7%)  hold  senior  executive  service  status 


GS  1 


GS5-8 


SES 


Pay  Grade  Classification 
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An  overwhelming  majority  of  responding  examiners  (70%)  are  under  forty  years 
of  age.   Another  twenty  percent  (20%)  reported  themselves  to  be  between  forty  and 
fifty.   Only  eleven  percent  (11%)  reported  themselves  as  over  fifty  years  of  age.   The 
chart  below  depicts  the  age  spread  of  respondents: 


Ages  20-29 


Ages  30-39 


Ages  40-49 
Age  Distribution 
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The  age  spread  of  respondents  was  included  as  a  useful  demographic  to 
instruct  analysis  regarding  computer  expertise  and  to  provide  a  statistical  tool  for 
assessing  whether  any  given  segment  of  examiners  --  such  as  those  who  entered  the 
corps  before  the  advent  of  computers  --  most  needed  training  and  skill  development. 
As  the  charts  below  illustrate,  both  computer  expertise  and  inexperience  are 
distributed  across  age  spectrums.   As  might  be  expected,  however,  examiners  in  their 
twenties  --  who  have  "grown  up"  with  computers  --  predominate  in  the  "computer  pro" 
category. 


Computer  Experience 


Novice 


Intermediate 


30-39  Years 


20-29  Years 


0-64  Years 


50-59  Years 


40-49  Years 


Age  Distribution  of  Those  Who  Described  Themselves 
as  Complete  Beginner  or  Novice 


20-29  Years 


60-64  Years 


30-39  Years 


-50-59  Years 


40-49  Years 


Age  Distribution  of  Those  Who  Described  Themselves 
as  Advanced  User  or  Computer  Pro 
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Of  the  entire  group  of  seven  hundred  and  twenty-four  (724)  respondents  to  the 
survey,  forty  percent  (40%)  are  classified  as  primary  examiners.   About  six  percent 
(6%)  of  all  respondents  reported  their  status  as  supervisory  patent  examiner.   These 
response  statistics  conform  generally  with  their  actual  distribution  throughout  the 
examining  corps. 
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Chapter  10:  CONCLUSIONS 


For  the  foreseeable  future,  the  examining  corps  of  PTO  will  depend  on  both 
manual  and  automated  tools  to  conduct  and  conclude  their  searches.    Dependency  on 
APS  is  growing  as  more  examiners  and  the  general  public  have  access  to  this 
technology.   Given  the  imminent  and  near  universal  availability  of  personal  computers 
at  examiners'  desktops,  this  particular  mode  of  accessing  APS  will  increase  both 
dependency  and  usage.   Within  the  not  distant  future,  remote  access  opportunities  -- 
for  both  examiners  and  the  public  --  will  make  APS  accessible  virtually  around  the 
globe  with  the  flick  of  a  switch. 

The  development  and  delivery  of  APS  to  both  examiners  and  the  public  has 
posed  considerable  challenge  to  the  Patent  and  Trademark  Office  and  to  the  vendors 
who  engineered  the  system,  deployed  its  products  and  now  support  it.    It  is  still  not 
perfect  and  much  work  is  yet  to  be  done  over  the  next  five  years.    But  APS'  current 
capabilities  represent  an  extraordinary  accomplishment  when  one  remembers  that 
none  of  this  technology  existed  a  decade  ago.   As  one  examiner  put  it,  "we  ought  not 
pillory  the  pioneers." 

In  order  to  make  the  program  fully  successful,  PTO  will  need  to  provide  a 
considerable  amount  of  on-going  review,  discipline  and  management  oversight 
regarding  how  the  system  will  be  expanded  and  supported.   Central  to  this 
responsibility  is  the  need  to  pay  close  attention  to  the  emerging  needs  of  its  primary 
users  —  the  examining  corps  and  the  public.   This  should  result  in  an  increased 
responsivity  to  the  performance-related  issues  raised  in  this  report. 

PTO  should  not  overlook  the  bedrock  reality  that  the  effectiveness  of  this 
technology  is  inescapably  linked  to  the  ability  of  real  people  to  use  it  well.    In  order  to 
take  full  advantage  of  the  resources  imbedded  in  the  technology,  examiners  ~  and  all 
users  -  must  be  trained  to  understand  and  use  these  electronic  tools  wisely. 

In  conclusion,  PTO  and  all  vendors  associated  with  this  project  have  much  work 
yet  to  do  as  user  needs  and  expectations  rise  with  the  tide  of  technological  changes 
yet  to  come.    But  for  now,  a  combination  of  anecdotal,  internal  reports  and  survey 
findings  strongly  support  the  conclusion  that  APS  is  currently  meeting  most  examiners' 
basic  business  needs. 
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Appendix  A 


Current  State-of-the-Art  Issues 


Three  areas  of  technology  advancement  afford  particular  opportunity  to 
enhance  the  productivity  of  examiners  using  APS.   They  are  the  increased  price 
performance  ratio  of  workstations,  the  maturity  of  the  client/server  architectural 
concept,  and  the  emergence  of  Standard  Generalized  Markup  Language  (SGML)  as  a 
widely  accepted  document  interchange  standard.   This  section  describes  the  current 
and  planned  activities  to  incorporate  state  of  the  art  technology  in  each  of  these  areas 
into  APS. 

Increased  Workstation  Price  Performance 

Providing  every  examiner  with  desktop  access  to  APS  has  always  been  a  PTO 
goal.    Until  now,  two  obstacles  have  delayed  achievement  of  that  objective.   The  first 
was  the  cost  of  the  hardware  required  to  effectively  meet  patent  examination 
requirements.   The  second  was  the  lack  of  a  workstation  operating  system 
environment  capable  of  smoothly  integrating  APS  with  office  automation  and  other 
desktop  applications  on  which  the  examiners  depend.    Both  of  these  obstacles  are 
rapidly  disappearing. 

The  continual  reduction  in  workstation  costs  now  makes  it  possible  to  configure 
an  affordable  desktop  hardware  platform  capable  of  running  APS  effectively. 
Experiments  are  underway  to  determine  the  most  cost  effective  display  monitor 
arrangement  for  desktop  deployment,  as  this  is  still  a  significant  cost  component. 
Recent  announcements  by  operating  system  vendors  provide  several  options  for 
integrating  APS  with  office  automation  and  other  desktop  applications.    Experiments 
are  underway  which  will  lead  to  the  selection  of  the  strategic  operating  system 
environment  for  desktop  deployment. 
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The  advancements  in  workstation  hardware  and  operating  system  technology 
also  allow  the  development  of  robust  high-performance  network  servers  capable  of 
performing  functions  previously  implemented  on  more  expensive  mainframe 
computers.   The  PTO  is  currently  developing  a  plan  to  migrate  the  image  database 
indexing  function  from  proprietary  software  operating  on  the  mainframe  to  COTS 
software  operating  on  a  network  server.1 

Client/Server  Architecture 

Client/server  architecture  as  a  preferred  model  for  network  configuration  has 
now  gained  almost  universal  acceptability.    In  such  environments,  one  dedicated 
computer  --  called  a  server  --  controls  all  the  programs  and  peripherals  that  are  part  of 
the  network.   Any  personal  computer  or  workstation  that  accesses  information  or  relies 
on  its  software  to  perform  tasks  it  cannot  do  on  its  own  is  called  a  client. 

This  arrangement  has  many  advantages.   Because  the  server  is  required  to 
function  as  the  real  "work  horse"  of  the  network,  much  of  the  computing  power  can  be 
centralized  in  one  space.   With  its  added  power,  the  server  can  perform  at  a  much 
higher  and  faster  level,  eliminating  the  need  to  empower  every  single  computer  or 
workstation  with  equal  capacity.   This  also  means  that  clients  can  operate  efficiently 
with  far  less  capacity  and  can  also  avoid  overloading  themselves  individually  with 
software  that  they  require  only  on  an  episodic  basis. 

The  maturation  of  the  client/server  architectural  concept  has  led  to  the  definition 
and  acceptance  of  standards  for  the  interface  between  clients  and  common  server 
functions.   As  a  result,  a  variety  of  commercial-off-the-shelf  (COTS)  products  exist  that 
implement  server  functions  in  a  standard  way.   Because  the  client  logic  that  manages 
the  user  interface  is  isolated  from  the  server  logic  that  manages  the  data  and  the 
complex  dependent  algorithms,  installations  can  make  substantial  changes  in  server 
technology  and  topology2  with  a  minimum  impact  to  end  users. 

The  initial  APS  design,  in  general,  followed  the  client/server  paradigm. 
However,  at  that  time  there  were  few  widely  accepted  standards  for  client/server 
interfaces  and  almost  a  complete  absence  of  COTS  products  relevant  to  APS.   Now 
that  client/server  topology  has  grown  to  such  widespread  acceptability  and  an  array  of 
useful  products  are  available,  PTO  has  two  key  efforts  underway  which  will 
incorporate  state-of-the-art  client/server  technology  into  APS. 


1The  ability  to  use  commercial-off-the-shelf  (COTS)  software  will  reduce  the  time  and  money 
required  to  develop  new  technology  and  greatly  accelerate  the  speed  with  which  cutting  edge 
enhancements  can  be  introduced  into  APS  and  other  office  automation  systems. 

2Topology  is  the  term  used  to  describe  the  physical  layout  of  a  local  area  network  (LAN). 
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The  new  SUS  deployment  of  APS  implements  a  rigid  separation  of  the  browse 
application  logic  that  manages  the  examiner  interface  from  the  logic  that  manages  the 
image  databases.   The  image  database  management  logic  is  implemented  in  an  APS 
subsystem  called  Generalized  Image  Service  (GIS).   The  browse  application  retrieves 
image  data  from  GIS  across  a  technology  independent  application  programming 
interface.   This  architected  partitioning  of  function  will  allow  the  PTO  to  procure  and 
install  advanced  COTS-based  servers  for  managing  image  data  with  minimum  impact 
on  the  browse  application  logic  and  the  examiners  with  which  it  interfaces. 

The  current  text  search  component  of  APS  is  not  implemented  within  a  client 
server  architecture.   The  bulk  of  user  interface  management  logic  executes  in  licensed 
software  on  the  mainframe.   This  has  impeded  the  ability  to  implement  improvements 
in  the  user  interface.   The  PTO  plans  to  migrate  APS  text  search  to  a  client/server 
architecture  and  has  specified  Z39.50  as  the  standard  protocol  for  client/server 
communications.   The  client  application  component  in  the  workstation  will  have 
complete  responsibility  for  managing  the  examiner  interface.   The  migration  of  text 
search  to  a  client/server  architecture  facilitates  the  following  improvements: 

Replacement  of  the  current  command  line  orientation  with  a  full  function 
graphical  user  interface 

More  rapid  inclusion  of  search  tools  and  other  productivity  enhancements 

Customization  of  the  user  interface  to  reflect  the  needs  of  specific  examiner 
constituencies 

The  display  of  patent  drawings  along  with  text 

The  routing  of  a  single  query  to  both  PTO  and  non-PTO  search  servers 

The  implementation  of  other  search  strategies  --  for  instance,  a  pattern 
recognition  search  engine  might  be  used  to  search  non-text  data  or  text  data 
generated  by  OCR  which  has  not  been  cleaned  up. 

The  initial  implementation  of  text  search  in  a  client/server  architecture  will  occur 
in  the  Global  Patent  Initiative  project. 
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Patent  Storage  in  SGML 

Numerous  PTO  projects  are  underway  to  identify  requirements  for  managing 
digital  image  access  and  are  now  being  consolidated  into  a  single  project.   This  effort, 
the  Generalized  Image  Service  (GIS)  project,  has  as  its  objective  the  development  of 
a  single  consistent  interface  that  can  effectively  support  all  current  and  future  projects, 
including  Classified  Search  and  Image  Retrieval  (CSIR)  and  other  image  retrieval 
applications  (e.g.,  PAM,  Global  Patents,  trademarks,  assignments,  etc.). 

An  important  goal  of  the  GIS  project  is  to  introduce  Standard  Generalized 
Markup  Language  (SGML),  which  will  enable  the  identification  of  separate  parts  of  a 
document.    By  tagging  the  parts  of  a  document  using  SGML  standards,  customers 
may  tailor  retrieval  and  viewing  of  the  document  to  their  specific  needs.   Over  the  next 
several  years,  worldwide  patent  offices  will  issue  and  interchange  patents  as  SGML 
tagged  documents.    SGML  will  facilitate  the  creation  of  a  full  copy  of  the  global  patent 
in  the  original  language  and  an  abbreviated  copy  containing  text  elements  that  have 
been  translated  into  English. 

A  patent  marked  up  in  SGML  will  be  a  compound  document  in  that  each 
component  will  be  stored  in  the  format  most  suitable  for  that  component.   Text 
components  will  be  stored  as  machine  processable  text.   Although  most  drawings  will 
be  stored  as  raster  images,  some  drawings  may  be  stored  in  machine  processable 
vector  format.    Equations  and  formulas  may  be  stored  in  specialized  machine 
processable  representations. 

The  markup  of  patents  in  SGML  facilitates  a  variety  of  user  interface 
enhancements.    For  instance,  while  a  drawing  is  displayed  in  one  window,  the 
examiner  might  scroll  through  items  of  text  that  reference  the  drawing  in  another 
window.   Notes  and  annotations  can  be  tied  to  specific  document  components. 

Currently,  the  PTO  maintains  about  a  million  bytes  of  data  for  the  average  U.S. 
Patent.    This  includes  a  complete  set  of  page  images  at  150  dpi  for  browsing,  a 
complete  set  of  page  images  at  300  dpi  for  printing,  and  a  copy  of  the  text  sections  for 
searching.   The  typical  SGML  patent  will  require  about  130  thousand  bytes  of  storage. 
The  conversion  of  pre-grant  and  weekly  issue  patents  to  SGML  format  will 
dramatically  reduce  the  rate  at  which  new  storage  servers  must  be  added  to  APS 
resulting  in  reduced  hardware  costs  and  less  operational  complexity. 

The  PTO  is  obligated  to  make  patents  available  to  the  public.   The  demand  for 
copies  of  patents  in  raster  image  format  is  limited  by  their  size  and  their  inability  to  be 
machine  processed.   Their  size  also  limits  the  feasibility  of  their  distribution  across  the 
Internet  or  other  wide  area  network.   The  reduced  size  and  increased  informational 
content  of  SGML  patents  will  offer  a  variety  of  new  options  for  public  access  and 
distribution. 
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The  APS  GIS  and  the  APS  display  server  is  currently  being  enhanced  to 
support  SGML  patents.   Prototyping  activity  is  planned  to  determine  the  appropriate 
changes  required  in  APS  to  display  and  print  a  mix  of  SGML  and  raster  image 
patents. 
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Appendix  B 


Schedule  of  APS  Patent-Related  Activities: 
FY1995-FY2000 


The  Patent  and  Trademark  Office's  Strategic  Information  Technology  Plan: 
FY1995-FY2000  describes  the  planned  enhancements,  replacements  and 
modifications1  of  the  automated  information  systems  and  subsystems  that  make  up  the 
Automated  Patent  System.   This  document  was  initially  produced  on  June  24,  1994;  a 
revised  version  was  released  on  October  24,  1994. 

The  following  is  a  brief  description  of  the  major  patent-related  system  design 
and  development  activities  planned  by  PTO  for  the  Automated  Patent  System. 

Classified  Search  and  Image  Retrieval  (CSIR) 

This  system  provides  the  basic  image  searching  capability  of  the  Automated  Patent 
System.    Introduced  originally  in  1988,  new  and  improved  features  have  been  added 
incrementally  as  new  technology  became  available.   At  present,  image  searching  is 
available  from  dual  screen  workstations  that  are  accessible  to  about  one-fourth  (22%) 
of  the  examining  corps.   By  the  end  of  this  decade,  virtually  all  examiners  will  have 
this  capability  at  their  desktop. 

In  the  meanwhile,  a  third  generation  shared-use  workstation  is  scheduled  to  be 
introduced  by  mid-March,  1995.   While  the  first  and  second  generation  workstations  -- 
introduced  during  the  1988-1992  period  --  were  entirely  custom-built,  the  third 
generation  equipment  is  a  hybrid  of  commercial  and  customer-built  hardware  and 
software.   This  latest  technology  will  be  available  at  forty-eight  (48)  shared-use 
workstations  and  supported  by  eight  (8)  group  printers. 


1Although  the  emphasis  in  the  document  is  on  outlining  projects  planned  for  the  period  FY1995- 
FY2000,  projected  and  actual  activity  and  product  milestones  are  described  for  all  tasks  and  products 
associated  with  PTO's  Automated  Patent  System  project  since  its  initiation  eleven  years  ago. 
Consequently,  you  will  see  FY1994  in  the  deployment  dates  following  each  project  described  below. 
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Desktop  Deployment  Project  -  FY1996-FY1999 

Desktop  access  to  both  image  and  text  searching  capability  was  one  of  the  most 
frequently  requested  enhancements  mentioned  by  the  examining  corps  --  both  during 
the  focus  group  discussions  and  in  the  survey  results.     PTO's  strategic  plan  calls  for  a 
complete  response  to  this  need  beginning  in  1996,  at  which  time  distribution  of 
commercially  available  CSIR  workstations  to  every  patent  examiner  and  classifier  will 
begin.   Through  these  desktop  workstations,  examiners  will  have  access  to  both 
image  and  text  search  capability  as  well  as  other  office  automation  services  scheduled 
to  be  introduced.    Upgraded  group  printers  will  support  this  expanded  capability  as  the 
project  is  fully  implemented  over  the  next  four  years. 

Examination  Toolbox  Project  -  FY1995-FY1999 

As  highly-individualized  research  requirements  emerge  from  different  art  groups,  some 
examining  groups  have  developed  "homegrown"  tools  to  facilitate  their  work.   (Work 
group  developed  software  designed  to  meet  local  application  requirements  is  a 
growing  trend  in  most  private  sectors  where  software  dependence  is  significant.) 
While  PTO  encourages  the  creativity  that  generates  such  products,  this  project  will 
regularize  how  actual  user  requirements  are  implemented  and  will  provide  an 
evaluation  framework  for  determining  whether  commercial-off-the-shelf  (COTS) 
products  or  those  developed  by  PTO  offer  the  best  solutions  to  examiner  needs. 

Toward  that  end,  the  Examination  Toolkit  Project  was  established  in  April,  1994  and 
follows  a  matrix  management  team  approach  which  involves  both  the  Offices  of  the 
Chief  Information  Officer  and  the  Assistant  Commissioner  for  Patents.   The  purpose  of 
this  project  is  to  develop  an  integrated  set  of  information  handling  desktop  tools  that 
can  facilitate  the  work  of  the  examining  corps.   Specifically,  the  Examination  Toolkit 
program  will: 

■  Identify  standards  to  be  used  in  tool  development  that  can  be  followed 
and  enforced  throughout  the  corps,  thereby  ensuring  broad-based 
application  of  such  tools  and  providing  a  basis  for  uniform  operations 
support; 

■  Determine  and  evaluate  the  need  for  specific  examination  tools,  thereby 
guarding  against  duplication  and  wasteful  proliferation; 

■  Evaluate  potential  products  able  to  satisfy  the  specific  needs,  thereby 
providing  a  framework  for  determining  what  tool  provides  the  best 
remedy  to  corps  needs;  and, 

■  Follow  LCM  methodologies  to  develop,  integrate,  test  and  deploy  tools, 
thereby  ensuring  the  reliability  and  transferability  of  such  tools  and 
standard  measurements  to  evaluate  performance. 
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Some  examples  of  tools  currently  being  considered  for  this  project  include 
electronic  manuals,  forms  creation,  image  browsing  tools,  text  search  assisting  tools, 
expert  help  systems,  communication  tools,  and  interface  tools  to  other  subsystems  on 
PTOnet. 

Patent  and  Trademark  Copy  Sales  (PTCS)  Project  -  FY1995-FY2000 

The  PTCS  is  a  standalone  order  entry  and  production  system  that  enables  both  the 
public  and  the  PTO  staff  to  place  orders  for  copies  of  patents  and  trademarks.   Order 
information  for  patent  copies  is  passed  to  the  APS  system,  which  retrieves  the  images 
of  the  patents  from  the  high  resolution  database.   Patent  copies  are  then  printed  out 
and  subsequently  processed  to  associate  them  with  a  specific  order  for  packaging  and 
distribution. 

This  five-year  project  will  support  the  maintenance  and  upgrade  of  PCTS,  including  all 
the  necessary  modifications  required  to  implement  the  Pre-Grant  Publication  (PGPub) 
system. 

Pre-Grant  Publication  (PGPub)  System  Project  -  FY1996 

This  new  project  requires  the  development  and  implementation  of  a  system  which  can 
ensure  electronic  access  to  patent  applications  eighteen  (18)  months  after  they  have 
been  filed  with  PTO.   When  complete,  the  system  will  incorporate  published 
application  data  into  PTO's  paper  and  electronic  search  files.   This  will  allow  more 
timely  access  to  the  contents  of  the  application  search  file  to  both  patent  examiners 
and  the  public. 

The  PGPub  system  -  which  will  provide  some  of  the  capabilities  associated  with  the 
Patent  Application  Management  (PAM)  system  --  will  eventually  be  incorporated  into 
PAM  if  the  PTO  is  able  to  proceed  with  the  project. 

Patent  Application  Management  (PAM) 

PAM  is  a  long-term  systems  development  program  which  will  provide  complete 
automation  of  the  application  process.  The  system  is  expected  to  simplify  the  corps 
search  activities  and  improve  communication  with  the  applicants  or  their  agents.   The 
system  will  also  provide  enhanced  tracking  of  patent  actions  throughout  all  phases  of 
the  application  search,  examination  and  issue  process.    Another  antecedent  system  -- 
PALM,  which  is  discussed  next  --  is  already  operational. 

Over  the  course  of  the  next  five  years,  three  major  on-going  activities  will  form  the 
evolutionary  bedrock  of  PAM  as  it  is  currently  envisioned.   They  are:  1)  Maintenance 
and  upgrade  of  the  current  PALM  system;  2)  Development  of  the  EASY  software;  and, 
ultimately,  3)  Development  of  the  PAM  system  itself. 
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Patent  Application  Location  and  Monitoring  (PALM)  System  Project  -  FY1994-FY2000 

PALM  currently  provides  workflow  tracking  and  status  reporting  for  all  patent 
application  processing,  including  application  file  location,  status,  title,  legal 
representation  and  other  statistics  about  examiner  production  and  docket  information. 
The  PALM  system  runs  on  a  mainframe  with  over  2,000  display  terminals,  bar  code 
readers  and  printers  directly  connected  to  the  mainframe. 

Between  now  and  the  year  2000,  the  PALM  system  will  continue  to  require 
enhancements,  maintenance  and  hardware  and  system  software  upgrade  activities. 
This  transitional  system  will  ultimately  be  replaced  when  the  new  Patent' Application 
Management  (PAM)  system  is  completed  in  the  year  2000.     Transitional  activities 
include  the  replacement  of  PALM  terminals  and  bar  code  readers  with  PCs,  which  will 
be  linked  to  PTOnet.   Changes  will  also  be  made  to  PALM  to  support  the  PGPub 
project  (see  previous  reference.) 

Electronic  Application  System  (EASY)  -  FY1995-FY1998 

The  Electronic  Application  System  (EASY)  project  will  provide  authoring  and  validation 
software  to  allow  patent  applicants  to  create  and  submit  electronic  patent  applications. 
This  software  will  serve  as  the  basis  for  electronic  application  filing  and  is  part  of  an 
international  alliance  which  includes  the  PTO,  the  European  Patent  Office  (EPO)  and 
World  Intellectual  Property  Organization  (WIPO).   When  fully  implemented,  the  EASY 
software  will  feature  streamlined  and  more  efficient  authoring  of  patent  applications, 
validation  of  the  application  package,  calculation  of  fees  and  electronic  signature. 

While  PTO's  objective  is  to  complete  the  EASY  project  by  the  year  2000,  current 
budget  levels  do  not  support  this  goal.   A  determination  has  not  yet  been  made  on 
whether  or  not  funding  for  this  project  will  be  increased. 

Patent  Application  Management  (PAM)  System  -  FY1995-FY2000 

PTO  now  relies  on  essentially  manual  procedures  for  processing  patent  applications 
(See  description  of  the  process  in  the  Introduction  section  of  this  report.)   PTO's  long- 
term  ambitions  for  fully  automating  the  application  process  through  PAM  are  far- 
reaching.   Among  their  stated  goals,  PTO  aims  to  reduce  the  cost  of  patent  application 
processing,  improve  quality  through  workload  and  process  management,  reduce 
patent  pendency,  facilitate  international  exchange  of  patent  information,  improve  the 
protection  of  intellectual  property,  improve  effective  access  to  PTO  information  by 
internal  users  and  the  public,  improve  management  reporting  capabilities,  improve 
security  and  improve  the  efficiency  of  patent  application  management. 
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PTO  has  approached  this  systems  development  project  through  reliance  on  a 
combination  of  in-house  and  contracted  resources.   Future  development  will  rely  on 
existing  and  planned  PTO  hardware  and  software  components  and  consolidation  of 
PAM  requirements  into  PTO-wide  acquisitions  of  other  components.   The  development 
of  the  PAM  system  will  be  contracted  for  under  the  System  Development  and 
Maintenance  (SDM)  contract  to  be  awarded  in  FY1996. 

The  FY1995-FY1996  phase  of  this  project  will  consist  of  preliminary  business 
processing  engineering,  functional  requirement  analysis,  basic  architectural  studies, 
along  with  data  and  activity  modeling.   Follow-on  phases  will  be  directly  shaped  by  the 
evolution  of  the  PGPub  system  (described  previously)  which  will  contain  many  of  the 
bedrock  features  required  by  PAM. 

Classification  Data  Systems  (CDS) 

This  system  maintains  classification  data  for  all  patent  and  application  documents. 
CDS  supports  the  US  patent  classification,  foreign  patent  classification,  issued  patent 
list  and  patent  file  maintenance.   From  1982  to  1992,  Power  Computing  provided 
remote  batch  processing  services  to  support  the  classification  function.    PTO  decided 
to  bring  the  classification  system  operation  in-house  in  1992  and  awarded  a  contract 
to  TAMSCO  to  continue  development  of  CDS. 

Placed  in  production  in  January,  1993  (FY1993),  CDS  version  1.0  supports  the  U.S. 
patent  classification  system  by  tracking  Miscellaneous  Transfers,  Weekly  Issues,  and 
Clerical  entry  of  Reclassification  project  data. 

CDS  Version  1.1  -  FY1995-FY1996 

This  project  involves  the  development  of  an  improved  software  for  CDS,  which  will 
provide  patent  classifiers  with  a  better  tool  to  support  the  U.S.  patent  classification 
system.    Desktop  microcomputers,  linked  to  APS  via  PTOnet,  will  be  provided  to  all 
classifiers.   CDS  1.1  will  provide  classifiers  with  a  number  of  new  features,  including 
improved  reclassification  tools,  cross  link  query  capabilities,  a  class  and  subclass 
reference  index,  and  access  to  domestic  and  international  search  aids. 

CDS  Version  2.0  -  FY1997-FY1999 

This  project  involves  the  upgrading  of  the  existing  CDS  hardware  and  the  CDS  1.1 
software  system.   Some  of  the  features  that  are  being  considered  for  development 
under  the  CDS  2.0  project  are:   international  search  tools;  a  dispute  decision 
database;  group  printers  for  the  Office  of  Classification  Systems;  the  addition  of 
desktop-accessible  images  to  classifier  functions;  matrix  classification;  automated 
foreign  language  translation  support;  and  the  complete  integration  of  the  CDS  1.1 
capability  to  add  drawings  to  definitions. 
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Text  Search 

The  APS  text  search  platform  allows  examiners,  classifiers  and  public  users  to  search 
and  retrieve  patent  text  information.   The  search  system  provides  a  customized  user 
search  interface  and  an  access  software  product  called  Messenger  --  licensed  to  PTO 
by  the  Chemical  Abstracts  Society.    In  addition  to  U.S.  patent  text,  Messenger 
accesses  PTO's  database  of  Japanese  patent  abstracts.    Messenger  customers  rely 
on  Classification  Data  Systems  (CDS)  for  relating  patents  with  class/subclass  data. 
Search  tools  --  such  as  concordances  and  examiner  search  procedures  and  guidelines 
--  support  various  other  text  search  activities. 

Because  of  the  large  costs  required  to  maintain  the  customized  Messenger  system, 
the  PTO  has  planned  an  accelerated  enhancement  of  the  current  text  search 
capability  (referred  to  as  the  Messenger  replacement  project).    In  addition,  PTO 
envisions  that  other  useful  applications  can  be  derived  from  the  Messenger 
replacement  program. 

- 

Messenger  Replacement  Project  -  FY1995-FY2000 

This  project  involves  the  replacement  of  the  customized  Messenger  software  system, 
which  is  currently  used  to  access  Text  Search,  with  a  commercially  available  software 
product.2   PTO  plans  to  issue  a  Request  for  Proposal  (RFP)  for  this  project  in  FY1997; 
a  contract  will  be  awarded  in  FY1998.   The  Messenger  Replacement  Project  will  also 
link  Text  Search  to  CDS,  PAM,  the  text  search  portion  of  Assignments,  Trademark 
systems,  external  databases,  and  the  Global  Patents  project. 

Global  Patents 

In  support  of  international  treaties,  examiners  currently  use  a  manual  process  for 
searching  foreign  patent  data  --  except  for  a  small  subset  of  Japanese  patent 
abstracts  which  examiners  may  access  through  the  current  automated  text  search 
system.   While  patents  issued  by  foreign  countries  have  become  increasingly 
important  in  establishing  the  validity  of  a  US  patent,  the  PTO's  ability  to  organize  and 
maintain  a  searchable  collection  of  foreign  patent  literature  is  currently  limited. 


2lf  an  appropriate  software  product  does  not  currently  exists,  PTO  plans  to  establish  a  partnership 
with  an  industry  leader  to  develop  a  software  product  that  would  eventually  become  commercially 
available. 
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Global  Patents  Project  -  FY1995-FY2000 

This  project  has  the  goal  of  introducing  an  integrated  system  that  will  allow  examiners 
to  access  European  and  Japanese  patents  from  their  desktop.   The  project  will  be 
implemented  in  an  incremental  fashion.    Initially,  examiners  will  be  able  to  search  and 
retrieve  an  English  language  abstract  of  the  Japanese  or  European  abstract,  as  well 
as  an  image  of  the  main  drawing.    Later,  the  full  images  of  the  first  few  pages  of  the 
patent  will  be  available  at  the  desktop.    Examiners  will  also  have  available  to  them  a 
full  array  of  foreign  patent  search  tools,  including  technology  classifications,  indices, 
thesauri  and  concordances.    PTO  also  plans  to  make  this  system  available  to  the 
public  through  the  Patent  and  Trademark  Depository  Libraries  (PTDLs). 

Non-Patent  Literature 

As  part  of  the  examination  search  process,  patent  examiners  must  search  existing 
literature  --  including  magazines  and  periodicals,  advertisements  and  books  ~  which 
have  been  cited  by  the  applicant  or  otherwise  available  to  the  public.   Currently,  some 
non-patent  literature  (NPL)  exists  in  the  paper  search  files  --  "the  shoes"  -  and  is 
routinely  used  by  examiners  in  their  searches.     However,  the  NPL  paper  search  file 
component  has  not  been  inventoried  and  little  is  known  about  the  characteristics  of  the 
documents  it  contains.    Examiners  also  have  access  to  a  collection  of  NPL  maintained 
by  the  Scientific  and  Technical  Information  Center  (STIC)  and,  by  extension,  to  outside 
institutions  which  support  the  Interlibrary  Loan  Service. 

Although  APS  has  no  internal,  automated  NPL  search  and  retrieval  capability,  it  does 
provide  an  interface  to  external  electronic  search  data  bases  which  contain  non-patent 
literature.   Also,  no  formal  business  process  exists  to  (a)  identify  NPL  which  should  be 
added  to  the  paper  search  files  in  a  structured  fashion;  (b)  maintain  an  inventory, 
similar  to  the  Master  Classification  File,  of  NPL  documents  which  have  been  placed  in 
the  search  file;  or  (c)  remove  NPL  from  the  paper  search  file  when  it  has  outlived  its 
usefulness. 

Non-Patent  Literature  (NPL)  Project  -  FY1999 

This  project  involves  the  development  and  deployment  of  NPL3  search  and  retrieval 
capabilities,  including  access  to  external  databases  and  the  incorporation  of  literature 
accompanying  a  patent  application.    PTO  will  begin  defining  the  scope  of  the  NPL 
Project  in  FY1 999. 


Examiners  can  find  NPL,  which  includes  periodical  and  journal  articles  and  books,  in  a  variety  of 
places.  Although  some  NPL  can  be  found  in  PTO's  paper  search  files,  this  material  has  never  been 
inventoried.   Examiners  also  access  NPL  through  external  databases  made  available  to  them  through 
the  Text  and  Image  Search  systems,  at  their  desktop  PC,  and  at  the  Scientific  and  Technical 
Information  Center  (STIC). 
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Appeals  Case  Tracking  System  (ACTS) 

The  Board  of  Patent  Appeals  and  Interferences  (BOPAI)  tracks  information  on  each 
appeals  case.   This  includes  production  information  for  each  of  three  Examiners  in 
Chief  who  review  the  case  and  decide  to  accept,  deny  or  accept  in  part  an  appeal.    It 
also  reports  the  decision  and  decision  date.    Implemented  in  1990,  the  Appeals  Case 
Tracking  System  (ACTS)  records  and  manages  this  information  using  30  terminals 
attached  to  an  ACTS  server  running  the  dBase  III  software  system.  However,  because 
of  the  extremely  low  budgetary  constraints  involved  with  the  development  of  the 
original  system,  the  reporting  capability  remains  limited. 

The  proposed  project  will  result  in  a  rewriting  of  the  original  system  to  take  advantage 
of  PTOnet  access  and  up-to-date  software  such  as  Paradox.   This  project  also 
includes  numerous  activities  aimed  at  correcting  the  problems  of  the  original  system. 
(It  should  be  noted,  however,  that  new  BOPAI  management  are  discussing  other  ways 
to  obtain  the  performance/production  reports  needed.) 

ACTS  Project  -  FY1995-FY1996 

This  project  involves  the  enhancement  of  the  Appeals  Case  Tracking  System,  which 
records  and  manages  information  on  appeals  cases  as  tracked  by  the  Board  of  Patent 
Appeals  and  Interferences  (BOPAI).   This  information  is  used  to  produce  reports  on 
production  and  performance.   Although  PTO  is  considering  other  methods4  to  produce 
the  necessary  BOPIA  reports,  it  is  currently  assumed  that  the  ACTS  Project  will 
continue  through  FY1996. 


4One  such  approach  would  link  the  report  requirement  to  the  PALM  system. 
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Appendix  C 


Definition  of  Terms 


APS  access  (Log-in  ID  and  Password):   Access  to  APS  is  controlled  by  two  sets  of 
numbers:   (1)  a  (constant)  log-in  ID  and  a  password  (which  periodically  must  be 
changed)  for  accessing  APS  through  image  workstations  (text  and  image  accessibility) 
or  single  screen,  shared-use  text  terminals  (text  only  accessibility);  and  (2)  an  entirely 
different  (constant)  log-in  ID  and  a  password  (which  periodically  must  be  changed)  for 
accessing  PTOnet  at  a  desktop  PC.   Some  patent  examiners  are  able  to  access  APS 
text  from  their  desktop  PC  through  modems  which  have  been  provided  by  PTO  or 
purchased  by  the  examiners  themselves. 

APS  coverage:   APS'  text  and  image  retrieval  capabilities  provide  patent  examiners 
with  access  to  two  distinct  sets  of  U.S.  patent  material.   Patent  examiners  with  access 
to  image  retrieval  are  theoretically  able  to  look  at  5.8  million  U.S.  patents  dating  back 
to  1790.    In  practice,  patent  examiners  find  that  some  patents  (or  pages  within  those 
patents)  are  "not  available"  to  them.   U.S.  patents  available  for  text  search  and 
retrieval  date  back  to  1971. 

APS  hours  of  operation:   APS  Text  and  Image  Search  is  now  available  to  examiners 
from  6:30  am  -  9:00  p.m.  Monday  through  Friday  and  from  7:30  a.m.  -  5:00  p.m.  on 
Saturday,  Sunday  and  some  holidays. 

Art  unit:   An  art  unit  is  a  subgroup  of  one  of  PTO's  sixteen  examining  groups. 
Examining  groups  are  divided  into  four  to  sixteen  art  units,  each  of  which  is 
represented  by  a  four  digit  number.   For  example,  art  unit  1302  {Food  or  edible 
material,  processes,  compositions  and  products)  is  one  of  the  eight  subgroups  which 
make  up  Group  1300  (Specialized  chemical  industries  and  chemical  engineering).   Art 
units  are  headed  by  a  Supervisory  Patent  Examiner  (SPE). 

Assignee  search  function:   In  an  APS  Text  Search,  one  index  or  specific  area  of  a 
patent  available  to  word  search  are  the  names  of  companies,  corporations,  or  persons 
to  which  patents  are  assigned. 
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Automated  Information  Systems  (AIS):    PTO's  automated  infrastructure  includes  a 
number  of  distinct  and  complimentary  automated  information  systems.    Many  of  the 
PTO's  management  information  and  day-to-day  operating  processes  are  also 
automated  and  dependent  on  the  availability  of  adequate  information  technology 
resources  for  support.    PTO's  AIS  serve  a  wide  variety  of  users  and  there  will  continue 
to  be  rapidly  developing  and  diverse  opportunities  to  employ  information  technology  to 
improve  how  the  PTO  conducts  its  business. 

Automated  Patent  System  (APS):   The  Automated  Patent  System  (APS)  is 
comprised  of  several  major  automated  information  systems  which  have  been  or  are 
being  developed  to  automate  all  aspects  of  PTO's  patent  operations.   The  systems 
which  are  projected  to  make  up  APS  in  the  future  include  the  Classified  Search-Image 
Retrieval  (CSIR)  system,  the  Classification  Data  Systems  (CDS)  system,  the  Text 
Search  system,  the  Patent  Application  Management  (PAM)  system,  and  the  Global 
Patents  System.   The  latter  two  systems  are  not  yet  operational.    PTO  is  planning  to 
add  two  additional  automated  information  systems  to  APS  in  the  future:   a  Non-Patent 
Literature  (NPL)  system  and  an  Appeals  Case  Tracking  System  (ACTS). 

Boolean  logic:  Boolean  logic  pertains  to  an  algebraic  combinatorial  system  treating 
variables,  such  as  propositions  and  computer  logic  elements,  through  which  multiple 
terms  can  be  searched  on  APS  using  the  operators  AND,  OR  and  NOT. 

"Break"  function  key:   The  "break"  function  key  on  the  cluster  workstation  terminals 
is  supposed  to  allow  a  user  to  terminate  an  executed  operation  before  it  is  completed. 

Business  Process  Reengineering  (BPR):   The  objective  of  Business  Process 
Reengineering,  a  management  technique  widely  used  throughout  the  private  sector,  is 
to  significantly  improve  the  efficiency  and  effectiveness  of  an  organization's  processes. 
PTO  recently  established  a  BPR  program  and  has  aggressively  pursued  its  expansion. 
In  1994,  a  newly  established  Office  of  Business  Process  Reengineering  began  the 
development  of  a  new  vision  for  PTO  operations. 

Center  for  Quality  Services  (CQS):    PTO's  Center  for  Quality  Services  provides 
guidance,  coordination  and  assistance  for  ensuring  quality  services  to  internal  and 
external  customers.   The  center's  activities  include  assistance  in  understanding 
customer  needs  and  expectations  so  that  quality  services  can  be  defined;  establishing 
service  standards;  measuring  customer  satisfaction;  benchmarking;  and,  determining 
the  policy  and  requirements  of  necessary  training  for  a  quality-driven  work  force. 
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Computer  Aided  Software  Engineering  (CASE)  tools:   Computer  Aided  Software 
Engineering  tools  are  used  to  support  the  implementation  of  the  information 
engineering  methodology  and  the  software  applications  that  automate  (at  least  in  part) 
a  particular  task,  such  as  development  of  documentation  or  writing  program  code. 
PTO  has  chosen  CASE  tools  to  assist  in  the  development  and  maintenance  of  its 
automated  information  systems.   These  tools  are  expected  to  aid  PTO  in  achieving 
specific  goals,  such  as:   to  focus  on  strategic  planning  and  business  goals  to  build 
systems  that  better  meet  PTO  "corporate"  needs;  to  better  integrate  information 
systems  by  using  shared  data  that  is  centrally  controlled;  and  to  stabilize 
representation  of  data  that  will  not  dramatically  change  as  business  procedures 
change. 

Chief  Information  Officer  (CIO):   PTO's  Chief  Information  Officer  is  the  principal 
advisor  to  the  Commissioner  of  Patents  and  Trademarks  on  the  evaluation  of 
information  technology,  the  architectural  design  of  automated  initiatives,  and  the 
development  of  strategic  information  technology  plans. 

Classification:   A  number  of  schedules,  definitions,  indices  and  other  search  tools 
have  been  developed  to  organize  U.S.  and  foreign  patents  and  non-patent  literature 
into  technical  categories  or  classifications.   These  classification  schemes  include  the 
U.S.  Patent  Classification  System;  classification  schemes  developed  by  foreign 
governments,  such  as  the  European  Patent  Office  (EPO)  and  the  Japanese  Patent 
Office  (JPO);  the  International  Patent  Classification  system;  and  the  classification 
schemes  developed  for  commercial  databases.    Every  patent  application  received  by 
PTO  is  assigned  a  class  and  a  subclass  number.   These  numbers  determine  which 
examining  group  and  art  unit  the  application  should  be  assigned  to  and  which  art  the 
examiner  should  search  to  make  a  patentability  decision.   The  U.S.  Patent 
Classification  System  is  continually  revised  and  patents  are  routinely  reclassified  in 
order  to  make  the  system  more  efficient  and  accurate. 

Classification  Data  Systems:   One  of  the  five  major  automated  information  systems 
that  make  up  the  Automated  Patent  System,  the  Classification  Data  Systems  (CDS) 
system  makes  it  possible  to  maintain  classification  data  for  all  patent  and  application 
documents  within  a  unified  database.   CDS  was  first  introduced  in  1982  and  updated 
in  1993.   An  incremental  enhancement  project  to  improve  hardware  and  software  is 
scheduled  for  deployment  over  the  period  FY1995-FY1999. 
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Classified  Search-Image  Retrieval  (CSIR):   One  of  the  major  automated  information 
systems  that  make  up  the  Automated  Patent  System,  the  Classified  Search-Image 
Retrieval  (CSIR)  system  provides  image  searching  of  over  5.8  million  patents  dating 
back  to  1790  on  dual  screen  workstations  to  four  of  the  sixteen  Patent  Examining 
Groups  or  approximately  23%  of  the  Patent  Examining  Corps.    PTO  currently  adds 
approximately  2,000  patents  to  the  CSIR  system  each  week.   The  CSIR  system  is 
comprised  of  the  Image  Search  database  and  the  CSIR  software.   CSIR  was 
introduced  in  incremental  phases  beginning  in  1988.   Desktop  deployment  to  all 
examiners  is  scheduled  to  begin  in  FY1996  and  be  completed  by  FY1999. 

Classifiers:     Classifiers  analyze  the  subject  matter  of  a  patent  application  and 
determine  its  technical  classification.   Classifiers  make  primary  (class)  and  secondary 
(subclass)  assignments,  which  are  primarily  based  on  the  U.S.  classification  system. 
They  are  also  responsible  for  reclassifying  patents. 

Client-server  architecture:    In  a  client-server  architecture,  each  application  is  divided 
into  two  parts:   a  client  task  communicating  with  the  end  user  and  a  server  task 
communicating  with  client  tasks  and/or  other  server  tasks.   The  client  task  executes 
the  front-end  user  interface  application  and  handles  only  the  data  needed  to  perform 
specific  functions.   The  server  task  performs  the  primary  data  processing  tasks  that 
need  to  be  shared  by  multiple  users  (clients).   There  can  be  many  types  of  servers, 
including  print,  facsimile,  electronic  mail,  computation,  application,  file  and  database 
servers. 

Cluster  rooms:   Cluster  rooms  house  groups  of  the  shared  use  image  workstations 
and  text  terminals,  and  are  located  throughout  the  areas  of  the  PTO  which  house 
patent  examiners. 

Cluster  workstation:   Also  called  an  image  workstation,  a  cluster  workstation  is  one 
of  a  group  of  workstations  located  in  a  single  room  or  cluster  that  allow  examiners  to 
access  the  CSIR  and  APS  Text  Search  systems.   Approximately  22%  of  the  Patent 
Examining  Corps  --  Groups  2200,  2100,  2300  and  1200  --  currently  have  access  to 
one  or  more  clusters  of  the  cluster  workstation. 

Commercial  databases:   There  are  a  number  of  commercial,  technical  --  notably 
CAS/STN,  Dialog,  Orbit,  Derwent,  Lexis,  Westlaw  -  databases  that  have  been  made 
available  to  certain  groups  within  the  examining  corps  and  even  to  certain  patent 
examiners  within  an  examining  group  based  on  their  technical  information  needs.    If 
patent  examiners  have  the  proper  account  number,  they  can  access  those  databases 
that  have  been  "offered"  to  them  at  several  locations:   from  their  desktop  PC;  from  a 
CSIR  cluster  workstation;  from  a  single  screen,  shared-use  text  terminal;  or  at  a 
terminal  in  the  Scientific  and  Technical  Information  Center  (STIC). 
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Commercial  Off-the-Shelf  (COTS)  software:   Commercial-off-the-shelf  software 
packages  are  commercially  available  software  packages  that  are  not  customized  to 
specific  user  needs. 

Concordance:   A  concordance  maintains  the  correlation  between  any  class  and  sub- 
class of  the  U.S.  Patent  Classification  System  and  the  corresponding  class  and  sub- 
class within  the  International  Patent  Classification  system. 

Configuration  Management  (CM):  A  configuration  management  program  improves 
an  organization's  ability  to  manage  and  control  the  various  components  of  computer 
hardware,  software,  and  documentation.   PTO's  CM  program  is  concerned  with  the 
baselining  of  automated  data  processing  (ADP)  resources  and  controlling  changes  to 
the  baseline  of  any  ADP  resource  under  the  CM  umbrella.    PTO's  automated  CM  tool, 
the  Product  Control  Management  System  (PCMS),  was  acquired  in  FY1994. 

Database  Management  System  (DBMS):   A  Database  Management  System  is  the 
information  engineering  methodology  and  the  software  applications  used  to  manage 
database  systems.   Where  cost-effective  and  feasible,  PTO  is  transitioning  to  a 
standard  DBMS. 

Desktop  PCs:    PTO  has  placed  some  type  of  personal  computer  (PC)  ~  386,  486  or 
Macintosh  --  on  the  desktop  of  almost  all  patent  examiners.   At  present,  not  all  of 
these  desktop  PCs  are  able  to  access  APS,  either  through  a  direct  dataline  connection 
to  PTONet  or  through  a  modem.   While  PTO  has  provided  modems  to  some 
examiners,  other  examiners  have  purchased  and  installed  modems  on  their  own. 

Document  Management  System  (DMS):   PTO  is  developing  the  Document 
Management  System  to  replace  PTO's  current  image  loading  software,  called  Off-Line 
Load  (OLL).   DMS  will  consist  of  generic  load  software  for  images  and  the  utilities 
necessary  to  maintain  and  report  on  the  status  of  all  APS  end-user  databases  --  both 
text  and  image.    DMS  will  also  provide  all  support  functions  for  text  and  image  data, 
including  software  tools  to  manage  data  stored  on  the  APS  system;  enhancement  to 
the  load  of  data  to  APS;  and  dissemination  of  the  data  resident  on  APS  to  the  public 
and  other  customers  on  demand. 

Electronic  lexicon:   An  electronic  lexicon  would  allow  examiners  to  cross  reference 
foreign  terms  with  their  English  equivalents. 

Electronic  personal  digest:   An  electronic  personal  digest  is  a  tool  to  allow  an 
examiner  to  put  together  an  electronic  collection  of  technical  material  --  such  as 
journal  articles  and  foreign  patents  --  to  facilitate  the  examiner's  individual  search 
process. 
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European  Patent  Office  (EPO):   The  European  Patent  Office  is  the  organization 
responsible  for  maintaining  patents  issued  by  European  countries. 

Examiner  Toolkit  Project:   The  Examination  Toolkit  Project  was  established  in  1994 
--  as  a  matrix  management  team  between  the  Office  of  Chief  Information  Officer  and 
the  Assistant  Commissioner  for  Patents  --  to  identify,  develop  and  deploy  in  a 
maintainable  configuration,  software  tools  which  assist  the  examination  process.    A 
patent  automation  strategy  that  specifically  tasks  the  Examination  Toolkit  Project  with 
improving  the  quality  of  the  search  process  by  providing  all  examiners  with  enhanced 
computer  functionality  at  their  desk  has  also  been  approved  by  the  Commissioner  of 
Patents  and  Trademarks. 

Examining  Group:  PTO's  Patent  Examining  Corps  is  divided  into  sixteen  technical 
examining  groups,  each  of  which  is  described  by  a  four-digit  number.  For  example, 
Group  1200  is  responsible  for  examining  patents  which  involve  Organic  Chemistry 
Drugs  and  Bio-Affecting  and  Body  Treating  Compositions,  while  examiners  in  Group 
1800  are  responsible  for  examining  patents  which  involve  Biotechnology.  Chemical 
examining  groups  begin  with  #1,  electrical  with  #2  and  mechanical  with  #3. 

File  integrity:    File  integrity  refers  to  the  completeness  of  the  manual  and  automated 
search  files.   The  reliability  of  any  given  search  is  dependent  upon  all  patents  being 
available  to  an  examiner.   Searches  in  the  manual  system,  "the  shoes,"  are  often 
compromised  because  all  patents  are  not  always  available.   This  is  particularly  true  in 
very  active  arts,  such  as  biotechnology  or  computer  systems.   While  examiners  are 
sometimes  given  a  "patent  not  available"  message  using  CSIR  or  Text  Search,  file 
integrity  in  general  is  much  greater  in  the  automated  search  systems. 

First  generation  workstation:   The  first  custom-built  CSIR  workstations  provided  to 
Group  2200  in  1988  and  then  to  Groups  2100  and  2300  in  1990  and  1991,  are 
referred  to  as  the  first  generation  workstations. 

Flip  rate:    Flip  rate  is  the  time  it  takes  to  go  from  page-to-page  or  document-to- 
document  using  CSIR. 

Focus  group:   A  focus  group  is  a  gathering  of  commentators  (in  this  case,  the  front- 
end  users  of  APS  --  the  patent  examining  corps)  to  elicit  their  opinion  on  a  given 
subject  (the  extent  to  which  APS  is  meeting  their  information  needs).    Focus  group 
sessions  are  typically  kept  small,  in  terms  of  the  number  of  people,  and  informal,  in 
terms  of  the  approach  to  engaging  people  in  the  discussion. 
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Foreign  art:    Many  patent  examiners  need  to  research  foreign  patents  for  a 
substantial  percentage  of  their  cases.    Foreign  text  can  be  accessed  through  several 
sources,  including  APS  text,  commercial  databases,  and  the  shoes.   The  only  foreign 
art  currently  available  on  APS  --  through  the  text-search-only,  shared-use  text 
terminals  --  is  Japanese  Patent  Office  (JPO)  abstracts  dating  back  to  1980.    Foreign 
images  are  currently  not  available  at  all  on  the  image  workstations.    Many  patent 
examiners  research  foreign  art  through  commercial  databases  that  have  been  made 
available  to  them  by  PTO.   One  of  the  primary  database  sources  for  foreign  patents  is 
Derwent,  a  London-based  service  that  provides  well-translated  patent  abstracts  from 
primarily  European  countries,  as  well  as  Russia.   Coverage  by  Derwent,  however,  is 
not  universal,  and  because  it  is  a  commercial  database,  it  is  not  available  to  the  entire 
Patent  Examining  Corps.   The  final  option  for  researching  foreign  art  is  to  go  to  the 
shoes.   There  are  separate  shoes  for  storing  foreign  art  under  the  corresponding  U.S. 
patent  class  and  subclass  shoes. 

Freestyle  search:  Freestyle  search  is  a  method  of  searching  text  in  a  database 
without  limiting  the  search  to  specific  fields,  such  as  title  or  abstract.  The  search 
query  is  applied  to  the  entire  database.   Lexis  uses  the  freestyle  method  of  searching. 

Fuzzy  logic:    Fuzzy  logic  is  a  multivalued  or  "vague"  logic  in  which  everything  is  a 
matter  of  degree,  including  truth  and  set  membership.   This  approach  is  in  contrast  to 
conventional  logic  which  treats  events  as  bivalent;  that  is,  they  are  either  true  or  false, 
in  Set  A  or  Set  (not  A)  --  but  not  both.    Fuzzy  logic  would  allow  for  an  event  to  be  in 
Set  A  and  Set  (not  A). 

Generalized  Image  Service  (GIS):   The  objective  of  PTO's  Generalized  Image 
Service  project  is  to  provide  a  standard  Application  Program  Interface  for  management 
of  and  access  to  image  data,  independent  of  data  format,  media,  and  location.   All 
projects  related  to  the  management  of  digital  image  access  will  eventually  be  brought 
under  a  single  GIS. 

Global  Patents:    One  of  the  five  major  automated  information  systems  that  make  up 
the  Automated  Patent  System,  the  Global  Patents  project  will  ultimately  make 
domestic  and  international  patent  information  available  to  the  examiner  on  the 
desktop.   This  system  is  expected  to  be  deployed  in  the  period  FY1995-FY2000.    PTO 
defines  global  patents  as  U.S.  patents  (text  and  image),  foreign  patents  (text  and 
image),  and  foreign  patent  search  tools. 

Group  printer:   Group  printers  are  connected  via  a  local  area  network  to  groups  of 
cluster  workstations  for  printing  selected  patents  or  text  search  information  accessed 
on  the  workstations. 
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"Help"  function  key:   The  "help"  function  key  on  the  cluster  workstation  terminals  and 
single  screen,  shared  text  terminals  is  supposed  to  guide  examiners  in  the  use  of  the 
text  search  and  image  search  systems. 

Help  line  (305-9000)  -  A  centralized  "help  line"  on  305-9000  --  or  the  Government 
Client  Support  Help  Line  as  it  is  officially  known  --  supports  all  user  automation 
problems.    PTO  is  currently  moving  away  from  a  centralized  help  line  structure  to  on- 
site  help  desks  at  designated  locations. 

High  density  optical  storage  devices  (juke  boxes):   PTO's  HDD  is  a  high  density 
optical  storage  device  consisting  of  a  Sony  Autochanger,  or  jukebox-like  peripheral, 
attached  to  a  network  server  designed  as  an  Intelligent  Controller  (IC).   Currently, 
there  are  seventeen  (17)  active  ICs  with  Autochanger,  each  capable  of  holding  fifty 
(50)  double-sided  optical  disks.   The  HDDs  store  image  data  at  300  dots  per  inch 
(DPI)  for  relatively  slow  speed/high  resolution  printing.   A  separate,  related  peripheral, 
the  rapid  access  device  (RAD),  is  used  for  faster  image  retrieval  tasks  (such  as 
browsing).    RADs  store  image  data  at  150  DPI. 

Image  workstations:   See  cluster  workstations. 

Information  Resource  Management  (IRM)  policy:    PTO's  recently  established 
Information  Resource  Management  (IRM)  policy  calls  for  the  centralization  of  technical 
direction,  information  technology  planning,  policy  formulation,  system  development  and 
acquisition,  business  process  improvement  and  IRM  Program  oversight  under  the 
Chief  Information  Officer.    Day-to-day  operational  management  of  the  PTO  information 
technology  infrastructure  will  be  under  the  direction  of  the  Associate  Commissioner 
and  Chief  Financial  Officer.   This  management  framework  will  provide  for  the 
evolutionary  application  of  new  technology  and  foster  uniformity  throughout  the  PTO 
through  standardization  of  hardware,  software,  and  data  to  the  maximum  extent 
possible. 

Interference  prints:    Interference  prints  are  drawings  of  all  pending  applications  kept 
in  a  file  by  subject  matter  which  are  to  be  searched  before  an  application  is  allowed  to 
locate  any  potentially  interfering  applications. 

International  Standardization  Organization  (ISO):   The  International  Standardization 
Organization  is  an  international  organization  based  in  Geneva.    Most  of  the  ISO's 
member  institutes  are  governmental  bodies  from  around  the  world.   The  ISO 
establishes  international  agreements  for  standards  on  products  and  commerce. 

Internet  Protocol  Suite  (TCP/IP):   The  Internet  Protocol  Suite  is  a  standardization 
protocol  for  accessing  the  Internet  ~  a  public  communications  network  used  in  over  30 
countries. 
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Japanese  Patent  Office  (JPO):   The  Japanese  Patent  Office  is  PTO's  counterpart 
agency  in  Japan. 

Life  Cycle  Management  (LCM):    In  January,  1994,  PTO  established  a  group  to  define 
and  institutionalize  a  standard  system  development  life  cycle  management  process.   A 
first  draft  of  an  LCM  Manual  was  completed  in  July,  1994  with  the  assistance  of  the 
Software  Productivity  Consortium. 

Local  printer:    Local  printers  are  connected  directly  to  a  cluster  workstation  or  single 
screen,  shared  text  terminal. 

Log-on/connect  time:    Log-on  or  connect  time  is  the  amount  of  time  it  takes  to 
access  any  aspect  of  the  Automated  Patent  System. 

Messenger:    Messenger  is  a  licensed,  proprietary  Boolean  search  software  that 
permits  word,  proximity  and  truncation  searching.    Developed  by  the  Chemical 
Abstract  Society,  Messenger  is  used  by  examiners  to  search  APS'  Text  Search 
system,  the  Classification  Data  System,  and  the  Japanese  Abstracts  database. 

Named  Document  Collection  (NDC):   A  Named  Document  Collection  is  a  set  of 
patent  images  that  an  examiner  has  pulled  up  during  a  search.    Examiners  can  save 
NDCs  on  APS  for  future  use. 

National  Information  Infrastructure  (Nil):   An  initiative  of  the  Clinton/Gore 
Administration,  the  objective  of  the  National  Information  Infrastructure  is  to  bring  vast 
amounts  of  information  to  the  public  via  a  seamless  web  of  communications  networks, 
computers,  databases  and  consumer  electronics.   The  Nil  initiative  sets  a  goal  to 
provide  the  broadest  possible  access  to  governmental  databases,  providing  both  the 
content  of  this  information  as  well  as  a  means  through  which  this  information  can  be 
distributed. 

National  Information  Infrastructure  (Nil)  Advisory  Council.   The  Department  of 
Commerce  has  formed  the  National  Information  Infrastructure  Advisory  Council  to 
address  the  technical,  policy,  and  privacy  issues  associated  with  Nil. 

Non-patent  literature:   Non-patent  literature  (NPL)  is  all  of  the  technical  literature  that 
is  used  in  the  patent  examination  process  that  is  not  a  U.S.  or  foreign  patent.    NPL 
includes  journal  articles,  books,  etc.    PTO  is  considering  the  development  and 
deployment  of  an  NPL  search  and  retrieval  system,  which  it  will  begin  defining  in 
FY1999. 

Office  action:   An  office  action  is  a  communication  from  the  examiner  with  respect  to 
the  merits  of  the  application  involved,  such  as  a  letter  setting  forth  rejections  or  a 
Notice  of  Allowance. 
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Office  of  Business  Process  Reengineering  (OBPR):   The  Office  of  Business 
Process  Reengineering  analyzes  the  Patent  and  Trademark  Office's  business 
enterprise  and  practices  through  the  application  of  business  process  reengineering 
techniques,  such  as  activity  and  simulation  modeling,  activity-based  costing,  and 
functional  economical  analysis.   The  Office  provides  central  guidance,  coordination, 
and  assistance  for  documenting  current  processes;  designing  new  processes  to  meet 
the  user  and  customer  requirements  for  Patent  and  Trademark  Office  products  and 
services;  researching,  analyzing,  and  recommending  policy  options  related  to  new 
processes;  and  developing  plans  for  the  transitioning  from  current  processes  to 
reengineered  processes. 

Office  of  Computer  and  Telecommunications  Operations  (OCTO):   The  Office  of 
Computer  and  Telecommunications  Operations  manages  the  PTO's  centralized 
computer,  data,  and  telecommunications  facility;  provides  maintenance  and  technical 
support  for  all  distributed  hardware  and  software  throughout  the  agency;  and  provides 
routine  modifications  to  meet  changing  program  requirements  and  improve 
performance  of  PTO's  operational  production  systems  and  communications  networks. 

Office  of  Information  Dissemination  (OID):   The  Office  of  Information  Dissemination 
develops,  maintains,  and  disseminates  a  diversified  portfolio  of  patent  and  trademark 
information  to  the  public.   The  OID  develops  electronic  information  products  and 
services;  distributes  patent  and  trademark  information  on  U.S.  technological  and 
economic  activities;  and  provides  information  dissemination  support  services  to  PTO 
customers. 

Office  of  System  Architecture  and  Engineering  (OSAE):   The  Office  of  System 
Architecture  and  Engineering,  an  office  within  the  Office  of  the  Chief  Information 
Officer,  develops  the  foundation  system  architecture  for  PTO's  information  technology 
infrastructure  and  plans  the  technological  evolution  of  the  Agency's  automated 
information  systems. 

Office  of  System  Development  and  Acquisition  (OSDA):   The  Office  of  System 
Development  and  Acquisition,  an  office  within  the  Office  of  the  Chief  Information 
Officer,  establishes  procedures,  technical  standards,  software  tools,  and  training  that 
supports  the  system  development  life  cycle  and  administers  the  project  management 
structure  for  the  development,  test,  implementation,  and  maintenance  of  the  PTO's 
AIS.   The  OSDA  acquires  information  technology  hardware  and  software  products  and 
services  consistent  with  PTO's  strategic  information  technology  plans  and  supporting 
information  technology  infrastructure. 

Oldest  effective  priority  date:   The  oldest  effective  priority  date  is  the  oldest  filing 
date  to  which  any  application  having  a  parent  application  is  entitled  priority  under 
either  35  USC  119  or  35  USC  120. 
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On-line  tutorial:   An  on-line  tutorial  allows  a  user  to  learn  how  to  operate  a  software 
program  while  actually  using  the  program. 

Open  system:   The  open  system  standards  define  the  format  in  which  data  is 
exchanged,  remote  systems  accessed,  and  services  invoked.   The  acceptance  of 
open  system  standards  supports  the  creation  of  system  architectures  that  can  be  built 
from  technology  components  available  from  many  vendors. 

Patent  examining  corps:   The  patent  examining  corps  is  made  up  of  professionals  in 
the  PTO  engaged  in  the  examination  of  patent  applications.    Members  of  the  patent 
examining  corps  examine  patent  applications  and  reissues  of  patents  to  determine  if 
the  applicants  are  entitled  to  patents.    Patent  examiners  also  process  reexamination 
and  statutory  invention  registration  requests  and  search  and  examine  International 
applications  filed  under  the  Patent  Cooperation  Treaty. 

Patent  and  Trademark  Copy  Sales  (PTCS):   The  Patent  and  Trademark  Copy  Sales 
system  is  an  automated  system  using  the  Classified  Search  and  Image  Retrieval 
system  (CSIR)  for  filling  requests  from  the  public  for  copies  of  patents. 

Patent  and  Trademark  Depository  Libraries  (PTDL):   The  Patent  and  Trademark 
Depository  Libraries  around  the  country  are  designated  libraries  which  receive  patent 
and  trademark  information  in  various  forms  to  enhance  access  of  patent  and 
trademark  information  to  the  general  public. 

Patent  Application  Location  and  Monitoring  (PALM)  System:   The  Patent 
Application  Location  and  Monitoring  (PALM)  system  is  PTO's  current  system  for 
tracking  the  location  and  status  of  a  patent  application  through  the  examination 
process. 

Patent  Application  Management  (PAM):   One  of  the  five  major  automated 
information  systems  that  make  up  the  Automated  Patent  System,  the  Patent 
Application  Management  system  will  fully  automate  the  patent  application  process, 
including  the  filing  of  a  patent  application  and  managing  the  application  through  PTO. 
This  system  will  eventually  replace  the  current  Patent  Application  Location  and 
Monitoring  System  (PALM)  and  is  expected  to  be  deployed  in  the  period  FY1995- 
FY2000  under  the  System  Development  and  Maintenance  (SDM)  contract. 

Patent  pendency:    Patent  pendency  is  the  amount  of  time  from  the  receipt  of  the 
application  to  application  issue  or  abandonment.   The  current  target  period  for  patent 
pendency  is  eighteen  months. 
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Patent  search  files:   The  patent  search  files  include  both  the  automated  --  through 
the  Automated  Patent  System  --  and  paper-based  --  through  the  shoes  --  files  of 
issued  patents. 

Peak  usage  periods:  Peak  usage  periods  are  those  times  of  the  day,  days  of  the 
week,  and  weeks  of  the  month  during  which  there  are  the  highest  number  of  users 
accessing  the  Automated  Patent  System. 

Pre-Grant  Publication  (PGPub):    In  response  to  Congressional  legislation,  PTO 
recently  decided  to  redirect  the  Patent  Application  Management  (PAM)  effort  towards 
the  development  and  implementation  of  the  Pre-Grant  Publication  system.    By  January 
1,  1996,  PTO  will  publish  patent  applications  within  18  months  of  the  earliest  filing 
date. 

Primary  Examiner:   A  patent  examiner  is  someone  who  has  completed  a  signatory 
review  process  and  has  been  granted  the  full  authority  to  finally  reject  applications  or 
allow  them  to  issue  as  patents  on  behalf  of  the  Commissioner. 

"Print"  function  key:   The  "print"  function  key  on  the  cluster  workstations  and  single 
screen,  shared  text  terminals  allows  users  to  request  a  printed  copy  of  data  or 
documents. 

Product  Control  Management  System  (PCMS):   Product  Control  Management 
System  is  an  automated  commercial-off-the-shelf  application  which  supports 
configuration  management  functions  for  identifying  components,  managing  changes  to 
components,  releasing  systems  versions,  and  relating  items  for  impact  analysis  and 
maintaining  synchronous  components.   PCMS  is  implemented  by  applying  the 
organization's  policies,  procedures,  methodology,  and  life  cycle  processes  through  the 
tool. 

PTONet:    PTONet,  or  the  PTO  Network,  is  a  comprehensive  end-to-end  data 
transmission  facility  linking  all  computers  in  the  PTO  --  from  the  largest  mainframes  to 
the  smallest  intelligent  terminals.   This  local  area  network  (LAN)  currently  connects 
computers  in  thirteen  buildings  in  Crystal  City,  providing  network  service  to 
approximately  3,900  customers  of  office  automation  products,  data  (electronic  mail 
and  document  sharing),  and  Patent,  Trademark,  and  other  business  applications 
stored  on  network  file  servers,  mainframes,  and  minicomputers.   The  implementation 
of  additional  applications  or  systems,  such  as  PAM,  Desktop  Workstation  Deployment, 
Patent  and  Trademark  Assignment  System,  and  Global  Patents  will  require  continuous 
enhancements  to  PTOnet  to  ensure  satisfactory  user  support. 

PTO  Newsletter:  The  PTO  Newsletter  is  a  regular  publication  which  aims  to  keep 
staff  informed  of  new  developments  in  PTO  operations  and  systems. 
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Refresher/update  training:    Refresher  or  update  training  is  considered  to  be  any 
training  an  examiner  receives  on  Image  Search,  Text  Search,  commercial  databases, 
or  Windows  after  their  initial  training  on  these  systems. 

Requirements  Traceability  Management  (RTM):     In  order  to  better  manage  the 
functional  and  performance  requirements  of  APS,  PTO's  Office  of  Chief  Information 
Officer  is  planning  to  acquire  an  automated  requirements  tool  called  Requirements 
Traceability  Management. 

Second  generation  workstation:   The  second  version  of  the  custom-built  CSIR 
workstations  that  were  provided  to  Group  1200  and  2300,  and  a  public  search  facility 
in  1993  are  referred  to  as  the  second  generation  workstations. 

Shared  Use  System  (SUS)  workstation:   The  third  generation  of  CSIR  workstations, 
which  were  supposed  to  be  installed  by  mid-March,  1995,  are  referred  to  as  the 
shared  ;se  workstations  (SUS).   The  SUS  workstations  are  a  hybrid  of  commercially 
availabu  and  custom-built  software  and  hardware  components. 

The  shoes:   A  manual,  paper-based  system  which  antedates  the  introduction  of 
automation,  the  shoes  are  comprised  of  wooden  and  metal  shoebox-sized  drawers 
which  contain  copies  of  patents  dating  back  to  the  time  patents  were  originally  granted 
by  the  U.S.  Government  (1790).   The  shoes  also  contain  a  limited  collection  of  foreign 
patents  and  non-patent  literature.   The  shoes  are  housed  throughout  the  PTO's 
complex  of  offices  in  Crystal  City,  Virginia. 

Standard  Generalized  Markup  Language  (SGML):   Standard  Generalized  Markup 
Language  tags  separate  parts  of  a  document  in  a  standard  way,  enabling  users  to 
customize  search,  retrieval  and  viewing  procedures  to  their  specific  needs.   Within  the 
next  several  years,  patent  offices  worldwide  --  including  PTO  -  will  convert  their 
patent  publishing  operations  to  SGML  and  issue  and  interchange  patents  as  SGML- 
tagged  documents.    SGML  will  facilitate  the  creation  of  a  full  copy  of  a  global  patent  in 
its  original  language  and  an  abbreviated  copy  containing  text  elements  that  have  been 
translated  into  English. 
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System  Development  and  Maintenance  (SDM)  contractor:    PTO  is  planning  to 
acquire  a  System  Development  and  Maintenance  contractor  to  succeed  the  current 
APS  contract,  increasing  the  scope  to  provide  PTO-wide  support  services.    By  the  end 
of  FY1996,  the  current  APS  contractor  and  several  other  contractors  will  be  replaced 
with  a  SDM  contractor.   This  new  contract  will  serve  as  PTO's  primary  source  for 
automation  support  services.   The  contract  will  be  used  for  all  planning,  analysis, 
design,  development,  enhancement,  and  maintenance  of  all  PTO  AIS  and  the 
supporting  technical  infrastructure.   Computer  center  operations,  network 
management,  and  office  automation  support  will  also  be  within  the  scope  of  this 
contract. 

Supervisory  patent  examiner  (SPE):   A  Supervisory  Patent  Examiner  is  a  senior 
Primary  Patent  Examiner  who  has  full  supervisory  responsibility  for  an  Art  Unit. 

System  recoverability:  System  recoverability  is  defined  as  the  extent  to  which  APS 
can  recover  work  performed  by  an  examiner  just  prior  to  a  system  outage,  boot-off  or 
lock-up. 

System  response  time:   System  response  time  is  the  amount  of  time  it  takes  APS  to 
respond  to  a  command  or  request. 

Systems  Acceptance  Testing/Independent  Verification  and  Validation 
(SAT/IV&V):   The  Systems  Acceptance  Testing/Independent  Verification  and 
Validation  process  comprises  all  functions  required  to  test,  integrate,  and  install 
software  enhancements  and/or  modifications  to  the  APS  product  environment.    Each 
element  of  this  activity  is  accomplished  within  the  framework  of  a  baseline  concept.   A 
baseline  consists  of  a  variable  number  of  software  capabilities  to  be  acceptance 
tested,  integrated,  and  installed  within  a  specific  time  frame. 

Target  search:   Target  search  is  a  non-Boolean  search  alternative  that  can  find  the 
most  relevant  articles  on  a  subject.   Dialog  uses  this  search  form. 

Text  Search:    One  of  the  five  major  automated  information  systems  that  make  up  the 
Automated  Patent  System,  the  Text  Search  system  allows  examiners  and  public 
customers  to  search  the  full  text  of  over  1.8  million  U.S.  patents  issued  since  1971 
and  the  English  translations  of  some  3.0  million  Japanese  Patent  Office  abstracts. 
This  search  tool  was  introduced  to  the  examining  corps  in  incremental  phases 
beginning  in  1986.   An  enhancement  project  to  replace  the  current  text  search  facility 
--  including  hardware  and  software  --  is  scheduled  for  deployment  in  the  period 
FY1995-FY2000.   The  Text  Search  system's  search  strategy  is  based  on  Boolean 
logic  structure. 
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Text  terminal,  shared,  single  screen:   The  single  screen,  shared  text  terminals  are 
used  by  Patent  Examiners  to  access  the  Text  Search  system.   These  text  terminals 
are  located,  typically  in  groups  of  four,  in  search  rooms  on  select  floors  of  the  PTO 
office  buildings  which  house  patent  examiners. 

Thesaurus/registry  feature:   A  thesaurus/registry  feature  would  allow  examiners  to 
pull  up  all  related  terms  and  spellings  of  a  word  during  the  search  process.   This  tool 
is  one  of  the  features  most  widely  desired  by  the  examining  corps. 

Training:   Training  includes  all  instruction  on  how  to  access  and  use  the  CSIR 
system,  the  Text  Search  system,  as  well  as  the  various  commercial  databases 
available  to  examiners.    Image  Search  training  is  provided  by  PRC.   Text  Search  and 
commercial  database  training  is  provided  by  the  Patent  Academy. 

Transportable  Image  Display  Workstation  (TIDW):   A  transportable  image  display 
workstation  is  any  shared  use  image  workstation  which  has  been  configured  to  be 
physically  moved  from  the  PTO  site  to  any  remote  location  and  function  to  resemble 
an  on-line  device  at  the  PTO.   The  TIDW  may  have  a  separate  optical  disk  drive  with 
sample  images  written  on  the  disk,  or  the  images  may  be  written  to  a  magnetic  hard 
disc  built  into  the  TIDW  to  simulate  an  on-line  image  retrieval.   The  TIDW  has  a  built- 
in  modem  which  may  be  used  to  dial  into  APS  for  on-line  access  to  text  search  and 
other  APS  system  functions  except  image  retrieval. 

U.S.  Patent  Classification  System:   The  U.S.  Patent  Classification  System  is  a 
scheme  for  organizing  patents  into  technical  categories. 

Walk-up  printer:   Walk-up  printers  have  no  search  capabilities  but  can  print  out 
copies  of  whole  patents.   They  are  located  throughout  the  offices  which  house  patent 
examiners. 

WinCOM  (for  hard  disk):   Communications  software,  including  graphical  user 
interfaces  (GUI)  which  allows  Patent  Examiners  to  operate  APS  in  a  more  user- 
friendly,  Windows-like  environment. 

WinNET  (for  the  PTONet):  Communications  software,  including  graphical  user 
interfaces  (GUI)  which  allows  Patent  Examiners  to  operate  APS  in  a  more  user- 
friendly,  Windows-like  environment. 

Workstation  rooms:   Workstation  rooms  house  the  shared  use  cluster  workstations 
and  text  terminals,  and  are  located  throughout  the  offices  which  house  patent 
examiners. 
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World  Intellectual  Property  Organization  (WIPO):   The  World  Intellectual  Property 
Organization  is  a  specialized  agency  made  up  of  125  members  of  the  United  Nations. 
WIPO's  objectives  are  to  promote  the  protection  of  intellectual  property  throughout  the 
world  and  to  ensure  administrative  cooperation  among  the  intellectual  property  unions. 
The  headquarters  for  WIPO  are  in  Geneva,  Switzerland. 

Xerox  Network  System  (XNS):   Xerox  Network  System  is  the  network  protocol 
previously  used  in  the  PTO.    It  has  been  replaced  by  TCP/IP  (Transmission  Control 
Protocol/Internet  Protocol). 
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Patent  pendency  has  risen 
slightly  in  the  period 
1989-1993  -by  6%,  but 
during  the  same  epoch,  the 
number  of  applications 
received  has  risen  by  1 5% 
in  the  period  1989-1993 
and  the  number  of 
decisions  rendered  has 
actually  risen  by  19%. 
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Workstation  Activity  In  Group  1 200  Clusters 

During  August  1 994 
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Workstation  Activity  In  Group  2100  Clusters 
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Workstation  Activity  In  Group  2200  Clusters 

During  August  1994 
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Workstation  Activity  In  Group  2300  Clusters 
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During  August  1994 
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Workstation  Malfunctions  In  Groups 

1200,  2100,  2200  and  2300  Clusters 

During  August  1994 
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Problem  Management  System  Reports 

The  Office  of  Computer  and  Telecommunications  Operations  (OCTO)  uses  the  Info/Man  problem  tracking  and 
management  system  to  monitor  the  resolution  of  identified  computer  hardware  and  software  problems,  and  to  determine 
areas  for  improving  service  to  our  customers. 

Problems  are  entered  into  the  system  via  two  means:  PTO  personnel  calling  the  help  desk,  and  PTO  personnel  who  have 
direct  access  to  the  problem  tracking  system.  Problems  are  classified  into  categories  such  as  Messenger,  CSIR,  Group 
printers,  office  automation  applications,  microcomputers,  TRAM,  PALM,  and  PTOnet.  The  problem  is  assigned  to  the 
appropriate  group  for  resolution.  Many  daily,  weekly,  and  monthly  reports  are  generated  from  the  information  that  is 
captured  in  this  system. 

In  keeping  with  recent  customer  focus  group  sessions,  OCTO  will  continue  to  use  this  system  to  analyze  support  of 
previously  agreed  upon  customer  service  standards  in  accordance  with  service  agreements  which  will  be  in  place  by 
October  1,  1994.  The  following  charts  provide  the  current  progress  toward  reaching  this  goal. 

Figure  ES-18  provides,  by  category,  the  total  number  of  problems  reported  in  August  1994,  and  those  which  were  earned 
over  from  the  previous  month.  There  are  two  reported  categories  which  require  futher  explanation.  The  category  labelled 
'PC'  provides  counts  of  PC  problems  that  are  not  related  to  an  application.  The  'Maintenance'  category  is  the  OCTO 
internal  workload  for  keeping  the  systems  updated  and  the  hardware  maintained. 

Figure  ES-19  provides  a  chart  of  the  time  required  to  resolve  customer  problems,  reported  by  category. 

Figure  ES-20  provides  the  average  time  for  resolution  of  problems,  reported  by  category. 


Figure  ES-18 
Problem  Records  by  Category 
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Problem  Management  System  Reports 


Figure  ES-19 
Problem  Resolution  Response 


100%7 

90% 
80% 
70% 


DAYS 


S§  31 

I 
plus  : 

0  21 

-  30  J 

B  11 

-  20 

Ma  ■ 

10 

Qo  - 

3 

Figure  ES-20 
Average  Resolution  Time 
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Transaction  Response  Times  for  Search  Applications 

Selected  monthly  average  response  times  for  the  PTO's  search  systems  are  shown  below.  The  average  response  time  for 
Messenger  search  transactions  for  the  last  1 2  months  was  21.8  seconds,  compared  to  1 8.2  for  the  previous  1 2  months.  (See 
Figure  BS-7).  The  values  shown  for  Classified  Search  and  Image  Retrieval  (CSIR)  represent  the  average  time  required  to 
retrieve  an  image  page  to  the  requesting  workstation.  The  average  response  time  for  the  last  12  months  was  0.5  seconds 
compared  to  0.8  for  the  previous  12  months.  (See  Figure  ES-8).  The  average  X-Search  response  time  for  the  last  12 
months  was  7.0  seconds,  compared  to  8.4  for  the  previous  12  months.  (See  Figure  ES-9). 


Figure  ES-7 

Messenger  Search  Request 

Monthly  Average  Response  Time 

Target:  80%  of  all  search  request  transactions  be 
completed  in  less  than  30  seconds.  Messenger 
response  time  in  August  1994  was  24.2  seconds. 
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Figure  ES-8 

CSIR  Image  Page  Retrieval 

Monthly  Average  Response  Time 

Target:  Average  of  all  image  page 

retrievals  to  be  under  1  second.  CSIR  response 

time  in  August  1994  was  0.44  seconds. 


Figure  ES-9 

X-Search  Internal  Transaction 

Monthly  Average  Response  Time 

Target:  80%  of  all  internal 

transactions  to  be  under  8  seconds. 

X-Search  response  time  in  August  1994  was 

5.9  seconds. 
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Messenger  Search  Requests  and  Response  Times 

There  were  2.3  million  search  requests  processed  by  Messenger  in  FY93  which  was  15%  greater  than  FY92.  During  the 
last  12  months  an  average  of  206,546  search  transactions  have  been  processed  each  month,  compared  to  187,620  for  the 
previous  12  months,  an  increase  of  10%.  During  this  same  period,  Examiner  usage  was  up  by  6%  and  public  usage  up 
26%.    A  peak  of  262,566  search  requests  was  reached  in  March  1994.    In  August  1994  the' Examining  Corps  initiated 
172,185  search  requests,  representing  74%  of  the  total  load.  Public  users,  which  include  two  PTO  Public  Search  facilities 
and  14  Depository  Libraries  (PTDL's),  initiated  5 1,013  requests  or  22%  of  the  total.  The  two  Public  Search  facilities 
initiated  38,791  requests  (17,318  at  PSR  and  21,473  at  CM1)  and  the  PTDL's  initiated  12,222  requests.  All  other  users 
accounted  for  10,502  search  requests.  (See  Figure  1-4). 

Figure  1-4 
Messenger  Search  Requests  by  User  Group 
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The  average  response 
time  for  the  last  12 
months  is  2 1 .8  seconds 
per  search  request, 
compared  to  1 8.2  for  the 
previous  12  month 
period.  Amdahl  6390-2 
storage  devices  were 
installed  in  May  1992, 
accounting  for  the 
improved  response 
times  in  Figure  1-5. 
Figure  1  -6  shows  the 
response  time 
distribution  for  August 
1994.  The  target  for 
Messenger  search 
response  time  is  for  at 
least  80%  of  all  requests 
to  complete  in  less  than 
30  seconds.  69%  of  all 
search  requests  for 
August  completed  in 
less  than  10  seconds, 
and  a  total  of  83  %  were 
completed  in  less  than 
30  seconds. 


Figure  1-5 
Messenger  Search  Requests  and  Response  Times  for  All  Users 
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Figure  1-6 
Messenger  Search  Request  Response  Time  Distribution 
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CSIR  Image  Page  Retrievals  and  Response  Times 

During  the  last  twelve  months  an  average  of  9  9  million  image  pages  have  been  retrieved  to  CS[R  workstations  each  month. 
Examiners  currently  account  for  61%  of  all  image  pages  retrieved,  averaging  6  1  million  pages  each  month  over  the  last 
twelve  months.  The  cumulative  number  of  image  pages  retrieved  by  examiners  in  FY93  was  68.5  million  which  was  28% 
greater  than  FY92.  A  comparison  of  the  last  12  months  against  the  previous  12  months  shows  a  88%  increase  in  pages 
retrieved  by  all  users  and  an  increase  of  34%  by  examiners.  A  Public  Image  Search  Facility  was  installed  in  July  1993, 
providing  public  access  to  CSIR.  Group  120  workstations  were  also  installed  during  the  same  period,  resulting  in  a 
significant  increase  in  the  number  of  pages  retrieved.  In  August  1994  there  were  6,740,81 1  image  pages  retrieved  by 
examiners,  4,927,129  pages  retrieved  by  public  users,  and  392,422  pages  retrieved  by  all  other  users. 


Figure  2-5 
Image  Pages  Retrieved  by  User  Group 
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The  target  response  time  for  image  page  retrievals  is  for  the  retrieval  of  each  page  to  be  completed  in  less  than  1  second. 
Figure  2-6  shows  the  monthly  average  for  the  last  12  months  was  0.55  seconds.  In  February  1992  the  average  retrieval 
response  time  reached  a  peak  of  1 .02  seconds.  The  average  response  time  for  August  1994  was  0.44  seconds  per  page. 


Figure  2-6 
Image  Page  Retrievals  and  Response  Times  for  All  Users 
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Messenger  Availability 

Figure  4-1  shows  Messenger  availability  for  the  last  24  months.  The  cumulative  Messenger  availability  for  the  most  recent 
12  month  period  was  99.7%.  The  cumulative  availability  for  the  previous  12  month  period  was  99.7%. 

There  were  no  incidents  which  affected  availability  of  Messenger  in  August  1994 


Figure  4-1 
Messenger  Availability  During  Scheduled  Operational  Hours 


100.00%^ 

99.00%4 

9e.oo%+ 

97  00%-f 

M.00%- 

H 

95.00%-t 

M.00%4 

93.00%+ 

92.00%4 

J 


Y~? 


II! 


m 


I 


m 


m 


/Gfl 


M- 


i 


:, 


HI 


CSIR  Availability 

Figure  4-2  shows  CSIR  availability  for  the  last  24  months.  The  cumulative  CSER  availability  for  the  most  recent  12  month 
period  was  99.7%.  The  cumulative  availability  for  the  previous  12  month  period  was  99.9%. 

There  were  no  incidents  which  affected  availability  of  CSIR  in  August  1994. 


Figure  4-2 
CSIR  Availability  During  Scheduled  Operational  Hours 
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PTOnet  Outages  Overview 

PTOnet  is  a  comprehensive  end-to-end  data  transmission  facility  linking  all  computers  in  the  PTO,  from  the  largest 
mainframes  to  the  smallest  intelligent  terminals.  PTOnet  comprises— 

A  local  area  network  (LAN) 

Currently  connects  13  buildings  in  Crystal  City,  providing  networking  services  to 
approximately  4,000  customers. 

The  LAN  is  divided  or  partitioned  into  smaller  segments  (101  segments  as  of  August 
1994)  to  lessen  the  impact  of  PTOnet  problems  upon  the  entire  PTO  if  a  segment  should 
fail.  That  is,  only  those  customers  attached  to  a  segment  will  be  impacted  if  problems 
occur;  the  remainder  of  the  PTO  may  continue  to  work  as  if  nothing  has  happened. 

Communications    devices  attached  to  the  network 

Synoptic  hubs,  like  the  hub  of  a  wagon  wheel,  allow  network  traffic  to  pass  through  to 
other  parts  of  the  network.  If  a  hub  experiences  problems,  more  than  one  segment  may 
be  affected. 

Routers,  as  the  name  implies,  route  or  direct  network  traffic  from  one  part  of  the 
network  to  another.  If  a  router  experiences  problems,  then  more  than  one  segment  may 
be  affected. 

Network  Servers  are  used  as  a  repository  for  software  applications  that  can  be  shared  by 
customers  on  the  network.  Servers  also  do  a  security  check  of  the  costomers  that  request 
access  to  the  network. 

Application    gateways  connect  customers  to  remote  locations  for  access  to 
commercial  databases,  the  Federal  Financial  System  (FFS),  the  National  Finance  Center 
(NFC),  and  internal  PTO  computers.  If  a  gateway  experiences  problems,  then  any 
customer  requiring  access  to  the  application  would  find  it  unavailable.  However,  all  of 
the  other  services  on  the  network  would  still  be  available  to  the  customer. 

User  drops  are  the  individual  outlets  in  each  customer's  office  that  enable  the  customer  to 
connect  their  computer  to  PTOnet. 

Network  services  such  as  office   automation  (e.g.,  word  processing,  electronic 
mail)  and  access  to  mainframe  applications. 

PTOnet  is  monitored  in  real  time  throughout  the  business  day.  When  network  problems  are  detected,  action  is 
taken  and  response  personnel  may  be  dispatched  to  the  problem  area  for  follow-up  action. 


Page  ES.8 
A-49 


PTOnet  Outages  -  August  *94 


Segments:        (A   segment   is  the   partitioning  of  the  network  wiring  to   accommodate  groups  of  users. 
There  were   101    network   segments   in  production   in  August) 
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Statistical  Analyses  Derived  from  Survey 
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Error  for  APS  sample  and  explanation  of  calculations 

The  error  rate  for  the  APS  sample  is  ±2.9  percent,  using  a  95  percent  confidence  standard.    This 
figure  is  based  on  724  completed  surveys  on  a  total  population  of  2028  supervisory  and  regular 
patent  examiners.    The  rate  is  calculated  by  using  two  formulas. 

na 
The  first  formula  is     n°~      na 

~~iv" 

•  N  is  the  actual  population.    In  this  instance,  it  is  2028. 

•  na  is  the  sample  surveyed,  in  this  instance,  724. 

•  nc  gives  the  sample  size  for  an  infinite  population  that  equates  to  sampling  724  out  of  a 
population  of  2028. 

The  formula  adjusts  the  sample  based  on  the  population  size.    It  is  necessary  because  the  formula 
for  calculating  error  rates  is  based  on  an  infinite  population.    We  know  that  error  rates  are  lower  for 
a  finite  population  versus  an  infinite  population.    For  example,  for  an  infinite  population,  a  sample 
size  of  2028  yields  an  error  rate  of  ±2.1  percent.    However,  the  entire  patent  examiner  population  is 
2028.    If  we  were  to  sample  2028  patent  examiners,  the  error  rate  would  be  zero,  lower  than  the 
rate  given  by  the  second  formula.    Error  rates  must  be  always  be  adjusted  to  account  for  a  smaller 
than  infinite  sample.    (If  the  population  is  large,  the  adjustment  produces  only  a  minor  difference.) 

The  second  formula  is     E2=zm^-n) 

•  nc  is  the  adjusted  sample  size  that  was  calculated  in  the  first  formula. 

•  z  is  1.96  for  the  95%  confidence  level. 

•  n  is  a  percentage  between  zero  and  100  percent  (0-1.00)    It  is  unknown,  but  50  percent  is 
used  to  give  the  most  conservative  estimate  of  error  rate.    tc(1  -  n)  is  highest  when  71=0.50; 
this  gives  the  highest  possible  error  rate  estimate.    When  7T=0.50, 

71(1  -  7l)=(0.50)(1.00  -  0.50)=(0.50)(0.50)=0.25.    Any  other  combination  yields  a  smaller 
number  and  therefore  a  lower  error  rate  (e.g.,  0.40  x  0.60=0.24;  0.30  x  0.70=0.21;  or 
0.25  x  0.75=0.19). 

•  E  is  the  calculated  error  rate. 

The  formula  calculates  the  error  rate  based  on  the  adjusted  sample  size.    As  sample  size  goes  up, 
the  error  rate  goes  down.    The  other  two  variables  (z,7C)  can  be  explained  as  follows: 

z  depends  on  the  confidence  level,  which  is  set  by  the  researcher:  95  percent  confidence  is  a 
standard.    It  is  a  measure  of  the  amount  of  certainty  needed  in  measurements.    The  higher 
the  confidence  level  set  the  more  certainty  needed;  therefore,  the  higher  the  z  and  the  higher 
the  margin  of  error. 

n  depends  on  survey  results.    It  represents  a  percentage  of  the  total  sample;  therefore,  it  is 
always  between  zero  and  one,  or  100  percent.    It  is  the  percentage  of  the  sample  that  gives  a 
certain  response.    For  example,  if  the  question  is  a  yes/no  question  and  40  percent  of 
respondents  answer  "yes",  n  is  0.40.    Since  we  have  many  questions  and  many  possible 
combinations  of  responses,  50  percent  is  used  to  give  the  most  conservative  (highest) 
estimate  of  the  maximum  error  rate.    Using  0.50  is  also  more  reliable  since  as  K  or  1  -  n 
approaches  zero,  the  formula  becomes  less  accurate.  A-52 
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How  satisfied  are  you  with  the  availability  of  your 
commercial  database(s)  when  you  need  to  use  it 
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How  satisfied  are  you  with  the  system  response  time 
required  to  execute  a  search  and  go  to  the  first  screen 
while  doing  a  search  of  your  commercial  data  base(s)? 
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How  satisfied  are  you  with  system  response  time 
required  to  go  screen-to-screen  while  doing  a  search 
of  your  commercial  database(s)? 
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How  satisfied  are  you  with  the  overall  user-friendliness 
(ease  of  operation)  involved  in  using  your  commercial 
database(s)? 
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How  satisfied  are  you  with  the  initial  training  you 
received  on  how  to  use  your  commercial  database(s)? 
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How  satisfied  are  you  with  the  amount  and  quality  of 
documentation  you  have  on  how  to  use  your 
commercial  database(s)? 
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To  what  extent  does  your  commercial  database  fulfill 
the  information  needs  of  your  job? 
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What  is  the  four  digit  number  of  the  art  unit,  i.e.,  the 
examining  subgroup,  you  are  currently  assigned  to  ? 
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How  would  you  describe  your  current  computer 
expertise? 
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How  long  have  you  worked  with  the  PTO  as  a  patent 
examiner? 
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Graduate  or  other  professional  degree 
(MA.MS.MBA.Ph.D,  etc) 

Note:  Due  to  the  multiple  response  nature  of  Question  59,  the 
frequency  distribution  is  divided  into  two  groups:  a)  Graduated  from 
college;  and  b)  Graduate  school  level. 
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Appendix  F 


Survey  Questionnaire  and  Cover  Letters 
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APS 
USER  CENSUS 

OCTOBER  1994 


\ 


1ATHI5 


&  ASSOCIATES 
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\ 


1ATI-II 


C 


&  ASSOCIATES 


October,  1994 


Dear  Internal  USPTO  APS  User: 


We  have  been  engaged  for  the  purpose  of  evaluating  how  well  the  USPTO's  APS  is  meeting 
the  needs  of  the  patent  examining  corps.  This  survey  is  one  part  of  a  multifaceted  review  of  the 
USPTO's  automated  systems.  You  may  also  be  invited  to  participate  in  other  aspects  of  this  review. 

Today,  we  are  inviting  you  now  to  give  us  feedback  about  your  expectations  and  satisfaction 
with  various  aspects  of  the  APS.  Your  feedback  is  very  important  and  your  responses  will  be  treated 
confidentially.  You  need  not  sign  your  name  or  provide  other  identifying  information.  Please  complete 
the  questionnaire  and  return  it  promptly  to  us  in  the  envelope  provided. 

Thank  you  for  taking  the  time  to  complete  this  questionnaire. 

Sincerely, 

Mathis  &  Associates 


All  Rights  Reserved.  1994  A"86 


OVERVIEW 


INSTRUCTIONS 


This  survey  is  intended  to  help  the  USPTO  assess  your  satisfaction  with  the  performance  of 
the  APS.  As  you  consider  your  experience  with  the  APS,  we  would  like  you  to  rate  how  well 
your  expectations  have  been  met  in  each  of  the  specified  areas  and  the  relative  importance 
of  each  element. 

The  survey  is  divided  into  five  parts: 

Overall  Information  Resource  Assessment 

•  Research  Tools  Assessment 

•  Future  Needs  Assessment 

•  Demographics 
Comments 

Your  open  and  honest  evaluation  is  important  and  we  will  safeguard  the  confidentiality  of  your 
responses.  Keep  in  mind  that  the  objective  of  this  study  is  to  gather  first-hand  impressions 
from  the  internal  USPTO  APS  users  so  that  your  constructive  feedback  can  be  integrated  into 
the  USPTO's  comprehensive  review  of  its  automated  systems. 

Under  the  Satisfaction  heading,  please  place  an  "X"  in  the  appropriate  circle.  Please  use  the 
following  scale: 

1  =  Very  satisfied 

2  =  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 
N/A  =  No  answer 

Under  the  Relative  Importance  heading,  please  indicate  how  important  you  feel  each  area 
is  to  your  ability  to  perform  your  job  by  placing  an  "X"  in  the  appropriate  circle.  Please  use 
the  following  scale: 

1  =  Critically  important 

2  =  Very  important 

3  =  Somewhat  important 

4  =  Not  too  important 

5  =  Not  at  all  important 

(As  you  apply  the  importance  scale,  keep  in  mind  the  adage  that  if  everything  is  of  equal 
importance,  no  element  is  more  important  than  any  other.  Try  to  establish  some  priorities 
about  your  needs  and  reflect  those  priorities  in  your  scoring.) 

A  few  questions  ask  you  to  indicate  frequency  of  use,  estimated  impact,  or  require  a  "yes" 
or  "no"  response. 

If  you  would  like  to  comment  further  on  a  question,  you  may  use  the  last  page. 
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OVERALL  INFORMATION  RESOURCE  ASSESSMENT 

In  this  section,  you  are  asked  to  indicate  your  satisfaction  with  various  aspects  of  the  Information  resources  available  to 
you  at  PTO.  In  addition,  please  indicate  how  important  each  element  is  to  the  performance  of  your  job.  If  you  do  not 
have  enough  experience  to  make  an  informed  evaluation,  select  N/A  for  "no  answer."  (Note  that  many  of  these  issues 
will  be  examined  in  a  more  detailed  way  further  along  in  this  questionnaire.) 


1 .    How  satisfied  are  you  with  the  following  aspects  of 
PTO's  information  resources? 


Satisfaction 
1=  Very  satisfied 

2  ■  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 


Relative Importance 
1  -  Critfcafly  Important 
2 -Very  Important 

3  »  Somewfwl  taportwf 

4  »  Not  too  Important 

5  *  Not  at  at  Important 


N/A    1      2      3      4     5     1      2      3 


a    Trie  initial  training  you  received  on  how  to  use  the 
computer  resources  (then)  available  to  you 


b.    The  refresher/update  training  you  have  received  on  how 
to  use  the  computer  resources  currently  available  to  you 


c. 


The  documentation  and  reference  materials  needed  to 
use  the  equipment,  systems,  and  databases  currently 
available  to  you 


d.    Your  access  (hook-up)  to  APS  and  other  databases 
needed  to  do  your  job 


e. 


Availability  and  maintenance  of  terminals,  PCs,  and 
CRTs  needed  to  do  your  job 


f.    Availability  of  printers  needed  to  do  your  job 


g.    Overall  cleanliness  in  the  shared  facility  workspace 


h.    Overall  lighting  in  the  shared  facility  workspace 


i.     Overall  climate  control  in  the  shared  facility  work- 
space 


j.     Effectiveness  of  the  Help  Desk  (305-9000)  in  resolving 
equipment  problems  or  systems  problems  you 
encounter  in  the  course  of  doing  your  job 


k. 


Availability  of  someone  who  can  help  get  you  through 
some  of  the  more  difficult  or  obscure  situations  that 
sometimes  arise  with  APS 


Effectiveness  of  the  PTO  Newsletter  in  keeping  you 
informed  about  system  changes,  new  features,  and 
how  to  use  them 


m.  Opportunities  for  swapping  information  among 
colleagues  about  search  methods  and  techniques 
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OVERALL  INFORMATION  RESOURCE  ASSESSMENT 

CONTINUED 


Satisfaction 
1=  Very  satisfied 

2  =  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 

2      3      4 


N/A    1 


Relative  Importance 
1 *  Crttcafly  Important 
2  °Very  Important 
\    3  »  Somewhat  Important 
<  *  Not  too  Important 
S  »  Not  at  afl  Important 

5     12      3      4      5 


n.   Accuracy  and  reliability  of  the  classification  system 
within  your  art  area,  in  order  to  get  the  proper  "hits' 

o.   Hours  of  operation  of  APS  and  the  walk-up  printers 

p.    Overall  reliability  of  APS  and  its  supporting  networks 

q.    Overall  user-friendliness  (ease  of  operation)  of  APS 

r.    Customization  of  APS  to  the  needs  of  your  examining 
group 

s.    Availability  of  a  channel  for  submitting  systems- 
related  suggestions  for  improvement 

t    Feedback  on  suggestions  submitted 

u.    Reliable  information  about  system  changes  and 
enhancements  scheduled  for  implementation 

2.   Have  you  received  initial  training  in  any  of  the  following  areas  -  and  if  so,  what  year? 

Yes 

Search  strategies  and  methods  for  your  art  area  in  1 9 


Text  search  on  APS 

Image  retrieval  on  an  image  workstation 

Use  of  one  or  more  commercial  databases 
(Consider  the  one  most  important  to  you.) 

WinCom/WinNet 

Other  (please  specify) 


in  19 
in  19 

in  19 
in  19 
in  19 


No 
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OVERALL  INFORMATION  RESOURCE  ASSESSMENT 

CONTINUED 


3.    How  would  you  rate  the  adequacy  of  your 
initial  training,  in  each  of  the  following  areas? 


N/A 


Never 
received  initial         Very 
training  on  this     adequate 


Generally  Generally  Very 

adequate         inadequate      inadequate 


a    Search  strategies  and  methods  for  your  art 
area 


b.    Text  search  on  APS 


c.    Image  retrieval  on  an  image  workstation 


d.    Use  of  one  or  more  commercial  databases 
(Rate  the  one  most  important  to  you.) 


e.    WinCorn/WinNet 


f.    Other  (please  specify) 


4.    How  much  of  your  initial  training  do  you  still 
remember? 


Never 
received  in  rial 
N/A       fraining  on  this      Most  of  it 


Almost 
Some  Very  We         nofiing 


a    Search  strategies  and  methods  for  your  art 
area 


b.    Text  search  on  APS 


c.    Image  retrieval  on  an  image  workstation 


d.    Use  of  one  or  more  commercial  databases 
(Rate  the  one  most  important  to  you.) 


e.    WinCorn/WinNet 


f.    Other  (please  specify) 
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OVERALL  INFORMATION  RESOURCE  ASSESSMENT 

CONTINUED 

5.   Have  you  received  refresher/update  training  in  any  of  the  following  areas? 

Yes 

Search  strategies  and  methods  for  your  art  area 
Text  search  on  APS 


No 


Image  retrieval  on  an  image  workstation 

Use  of  one  or  more  commercial  databases 
(Consider  the  one  most  important  to  you.) 


WinCom/WinNet 


Other  (please  specify) 


6.  How  would  you  rate  the  adequacy  of  your 
refresher/update  training,  in  each  of  the 
following  areas? 


Never 
received  intfal         Very  Generally        Generally  Very 

N/A        taining  on  this      adequate         adequate        inadequate      inadequate 


■     -"•"-..-:-"                                   ■  ,'  "  ' 

a   Search  strategies  and  methods  for  your  art 

area 

■:■:*>:*: v1**'?                                   '    '     ..-•.•:',•;■':.■  .    '  :'.  " '-'.'. ■  '  ;'.>'    ,           *"* 

b.   Text  search  on  APS 

c.    Image  retrieval  on  an  image  workstation 

d.    Use  of  one  or  more  commercial  databases 
(Rate  the  one  most  important  to  you.) 

e.    WinCom/WinNet 

f.    Other  (please  specify) 
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OVERALL  INFORMATION  RESOURCE  ASSESSMENT 

CONTINUED 


7.    How  would  you  rate  the  adequacy  of  docum  enta- 
tion  and  reference  materials  in  each  of  the  following 
areas? 


Very  Generally         Generally  Very 

N/A  adequate  adequate        inadequate       inadequate 


a    Search  strategies  and  methods  for  your  art  area 


b.    Text  search  on  APS 


c.    Image  retrieval  on  an  image  workstation 


d.    Use  of  one  or  more  commercial  databases  (Rate 
the  one  most  important  to  you.) 


e.    WinCom/WinNet 


f.    Other  (please  specify) 


8.    How  satisfied  are  you  that  you  have  the  training  and  skills  needed  to  fully  utilize  the  information  resources  available  to 
you  at  PTO? 


Very 

Generally 

Generally 

Very 

safsied 

safsfed 

Neutral 

dissatisfied 

cfissatisSed 
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RESEARCH  TOOLS  ASSESSMENT 

In  this  section,  you  are  asked  to  describe  your  level  of  usage  of  the  various  channels  used  to  access  APS  and  how 
well  each  of  those  channels  is  performing.  If  you  do  not  have  access  to  a  particular  channel,  select  N/A  for  "no 


access. 


9.    How  would  you  describe  your  level  of  usage  of  the 
following  PTO  research  channels? 


1  =1  use  it  fa  nearly  every  application 
2=1  use  it  for  most  applications 
3=1  use  it  fa  some  applications 
4=1  use  it  very  infrequently 
5=1  do  not  use  it 

Nearly  every        Most  Some  Very  Do  not 

appljcafion     applicafons  appticafons  infrequently        use 

N/A  1  2  3  4  5 


a    The  shoes 


b.    Text  search  through  a  shared,  single  screen  text 
terminal 


c.    Text  search  through  an  image  workstation 


d.    Text  search  through  my  desktop  PC 


e.    Image  search  at  an  image  workstation 


f.    Commercial  databases  at  a  special  center  (or  a 
shared  facility  workspace) 


g.    Commercial  databases  through  my  desktop  PC 


h.    Library  reference  volumes  (journal  articles,  foreign 
patents,  etc.) 


i.    Walk-up  printers 


j.    Printers  in  the  shared  facility  workspace 


k.    Printer  at  or  near  my  desktop 
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RESEARCH  TOOLS  ASSESSMENT  CONTINUED 

Shoe  Search 

1 0.  Do  you  have  access  to  the  shoes  in  the  course  of  performing  your  job? 


Yes 


No  (skip  to  quest  on  14) 


1 1 .  How  would  you  describe  your  amount  of  usage  of  the  shoes  during  the  past  three  months? 

Very  heavy  (nearly  Heavy  (one  or  more  Medium  (1-3  times  Light  (less  than  None 

everyday)  times  per  week)  permonlh)  once  per  month)  (skip  to  question  13) 


12.  How  satisfied  are  you  with  the  following  aspects  of  the 
shoes? 


Satisfaction 
1=Varysat»slUd 

2  ■  Genaraly  satisfied 

3  =  N«utral 

4  =  GsnaralyoTs«at)*fl»d 

5  =  Vary  dissatisfied 


f  eCrtfeafytaportart 

2  »  Vary  Important 

3  =  SoBMwrtiat  Important 
4 -Not  too  important 
5«Mot*al  important 


1234512345 


■. 


a    Need  to  travel  or  move  around  to  get  the  material  you 
need 


b.    Overall  cleanliness  and  ease  of  working  in  the  shoe 
area 


c.    File  integrity  (completeness  of  expected  items) 


d.    Speed  with  which  you  can  search  the  shoes  and 
locate  needed  material 


13.  To  what  extent  do  the  shoes  fulfill  the  information  needs  of  your  job? 


Completely 


To  a  large  extent        ;      To  some  extent 


Very  little 


Not  at  all 
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Text  search  through  a  shared,  single-screen  text  terminal 

14.  Do  you  have  access  to  text  search  through  a  shared,  single-screen  text  terminal? 


Yes 


No  (skip  to  question  18) 


15.  How  would  you  describe  your  amount  of  usage  of  text  search  through  a  text  terminal  during  the  past  three  months? 


Very  heavy  (nearly 
everyday) 


Heavy  (one  or  more 
limes  per  week) 


Medium  (1-3  limes 
per  month) 


Light  (less  than 
once  per  month) 


None 
(skip  to  question  17) 


16.  How  satisfied  are  you  with  the  following  aspects  of 
text  search  through  a  shared  text  terminal? 


Satisfaction 
1=  Vary  satisfied 

2  =  Generally  satisfied 

3  =  Neutra! 

4  =  GeneraBy  dissatisfied 

5  =  Very  dssattefled 

12      3      4     5     1 


Relative  Importance 

1-Crt8cafly  important 
Z  *  Vary  Important 

3  *  Somewhat  Important 

4  =  Not  too  Important 
5=  Not  at  aB  important 

2     3     4     5 


a    Accessibility  of  the  shared  text  terminal  equipment  to  your 
work  area 

b.    Availability  of  shared  text  terminal  equipment  when  you 
need  to  use  it 

c.    Operability  and  maintenance  of  terminals  and  CRTs  in  the 
shared  text  terminal  area 

d.    Operability  and  maintenance  of  printers  in  the  shared  text 
terminal  area 

e.    Log  on/connect  time  required  to  initiate  text  search 
through  a  shared  text  terminal 

f.    System  response  time  required  to  execute  a  search  while 
doing  text  search  through  a  shared  text  terminal 

g.    System  response  time  required  to  go  screen-to-screen  or 
patent-to-patent  while  using  a  shared  text  terminal 

h.    Overall  user-friendliness  (ease  of  operation)  involved  in 
using  a  shared  text  terminal 

i.    System  recoverability  (ability  to  save  work  following  a 
system  crash,  lock-up  or  boot-off)  while  using  a  shared 
text  terminal 

17.  To  what  extent  does  text  search  through  a  shared  text  terminal  fulfill  the  information  needs  of  your  job? 
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Completely 


To  a  large  extent 


To  some  extent 
9 


Very  little 


Not  at  all 


Textsearc 


>rkstation 


18.  Do  you  have  access  to  text  search  through  an  image  workstation? 


Yes 


No  (skip  to  question  22) 


19.  How  would  you  describe  your  amount  of  usage  of  text  search  through  an  image  workstation  during  the  past  three 
months? 


Very  heavy  (nearly 
everyday) 


Heavy  (one  a  more 
times  per  week) 


Medium  (1-3  times 
per  month) 


Light  (less  than 
once  per  month) 


None 
(skip  to  question  21) 


20.  How  satisfied  are  you  with  the  following  aspects  of 
text  search  through  an  image  workstation? 


Satisfaction 
1=  Very  satisfied 

2  =  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 


Relative  Importance 
I^CrWcafly  Important 

2  >  Very  Important 

3  «  Somewhat  Important 

4  *  Mot  too  Important 

5  =  Not  at  aS  Important 


12      3      4     5     12      3      4      5 


a    Accessibility  of  the  im  age  workstation  to  your  work  area 


b.    Availability  of  image  workstation  equipment  when  you 
need  to  use  it 


c.    Operability  and  maintenance  of  terminals  and  CRTs  in  the 
image  workstation  area 


d.    Operability  and  maintenance  of  printers  in  the  image 
workstation  area 

e.    Log  on/connect  time  required  to  initiate  text  search 
through  an  image  workstation 

f.    System  response  time  required  to  execute  a  search  while 
doing  text  search  through  an  image  workstation 

g.    System  response  time  required  to  go  screen-to-screen  or 
patent-to-patent  while  doing  text  search  through  an 
image  workstation 

h.    Overall  user-friendliness  (ease  of  operation)  involved  in 
doing  text  search  through  an  image  workstation 

i.     System  recoverability  (ability  to  save  work  following  a 
system  crash,  lock-up  or  boot-off)  while  doing  text  search 
through  an  image  workstation 
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21.  To  what  extent  does  text  search  through  an  image  workstation  fulfill  the  information  needs  of  your  job? 


Completely 


To  a  large  extent  To  some  extent 


Very  little 


Not  at  all 


Text  search  at  a  PC  on  or  near  your  desktop 

22.  Do  you  have  a  PC  on  or  near  your  desktop? 


Yes,  one  PC 


Yes,  more  than  one  PC  No  (skip  to  queslion  29) 


23.  What  kind  of  PC  is  it?  (Check  as  many  as  apply.) 

";;■    286  J|    386 


Mac 


24.  Is  your  PC  currently  hooked  up  to  APS  text?  (Check  as  many  as  apply.) 


Yes,  via  PTO  Net 

Yes,  via  modem 

No,  the  PC  has  no  hookup  to  APS  text  (Please  state  here  the  reason  why,  if  you  know,  then  skip  to  question  29): 


25.  Is  your  PC  currently  using  WinCom  or  WinNet? 


Yes,  WinCom 
Yes,  WinNet 
No,  none  of  the  above 
Not  sure 


26.  Is  your  PC  hooked  up  to  a  working  printer  somewhere  in  your  local  work  area? 


Yes 


No  (skip  to  question  29) 
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27.  How  satisfied  are  you  with  the  following  aspects  of 
text  search  through  your  desktop  PC? 


Satisfaction 
1=  Very  satisfied 

2  =  GeneraBy  satisfied 

3  =  Neutral 

4  =  GeneraBy  (Bssattsfled 

5  =  Very  dssattsfled 

1      2      3      4     5 


-:;  Relative  Importance  .-■  .^H: 
n  onosiy  mpanam 

2  «  Very  Important 

3  jomawhitlrrifjfjtaiit 
*<*  Hot  too  important 

5  =  No<ttafltepa1afrt 

12      3      4     5 


:*&S: 


II II I  III  ll."M|"|MMHIWW 


a    Log  on/connect  time  required  to  initiate  a  text  search 
through  your  desktop  PC 


b.    System  response  time  required  to  execute  a  search 
while  doing  text  search  though  your  desktop  PC 


IIIIIHimeHMMimMIIMIMIHHHIHIHIMaC 


c.    System  response  time  required  to  go  screen-to- 
screen  or  patent-to-patent  while  doing  text  search 
through  your  desktop  PC 


d.    Overall  user-friendliness  (ease  of  operation)  involved 
in  doing  text  search  through  your  desktop  PC 


itMHHitmmmmoKiwiMiii  um  tin  milium:- 


e.  System  recoverability  (ability  to  save  work  following 
a  system  crash,  lock-up  or  boot-off)  while  doing  text 
search  through  your  desktop  PC 


28.  To  what  extent  does  text  search  through  your  desktop  PC  fulfill  the  information  requirements  of  your  job? 


Completely 


To  a  large  extent  To  some  extent 


VeryDtOe 


Not  at  all 


Image  Retrieval  through  an  Image  Workstation 

29.  Do  you  have  access  to  image  retrieval  through  an  image  workstation? 


Yes  (skip  to  question  31) 


No  (answer  question  30,  tien  skip  to  question  34) 


30.  Based  on  what  you  know  now,  would  you  want  access  to  an  image  workstation  if  you  could  have  it? 


Definitely  yes 


Probably  yes  Not  sure  Probably  no 

(Now  skip  to  question  34) 


Definitely  no 


31 .  How  would  you  describe  your  amount  of  usage  of  image  retrieval  through  an  image  workstation  during  the  past  three 
months? 


Very  heavy  (nearly  Heavy  (one  a  more  Medium  (1  -3  times 

every  day)  times  per  week)  per  month) 


Light  (less  than  None 

once  per  month)  (Skip  to  question  33) 
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32.  How  satisfied  are  you  with  the  following  aspects  of  image 
retrieval  through  an  image  workstation? 


Satisfaction 

1  =  Very  Mttefled 

2  =  Generally  satisfied 

3  =  Neutral 

4  =  Generally  dissatisfied 

5  =  Very  dissatisfied 

2      3      4     5 


Relative  Importance 

1  *  Crftkafly  important 

2  *  Very  Important 

3  «  Somewhat  important 

4  £  Not  too  important 

5  -  Not  at  aS  important 

2     3     4     5 


a    Accessibility  of  the  image  workstation  to  your  work  area 

b.   Availability  of  image  workstation  equipment  when  you 
need  to  use  it 

c.    Operability  and  maintenance  of  terminals  and  CRTs  in  the 
image  workstation  area 

d.    Log  on/connect  time  required  to  initiate  image  retrieval 
through  an  image  workstation 

e.    System  response  time  required  to  go  to  first  patent  while 
doing  image  retrieval  through  an  image  workstation 

f.    System  response  time  required  to  go  screen-to-screen  or 
patent-to-patent  while  doing  image  retrieval  through  an 
image  workstation 

g.    Overall  user-friendliness  (ease  of  operation)  involved 
in  doing  image  retrieval  through  an  image  workstation 

h.    System  recoverability  (ability  to  save  work  following  a 
system  crash,  lock-up  or  boot-off)  while  doing  image 
retrieval  through  an  image  workstation 

33.  To  what  extent  does  image  retrieval  through  an  image  workstation  fulfill  the  information  needs  of  your  job? 


Completely 


To  a  large  extent 


To  some  extent 


Very  little 


Not  at  all 
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FUTURE  NEEDS  ASSESSMENT 

In  this  section,  we  are  asking  you  to  predict  the  likely  positive  impact  on  your  job  performance  if  certain  features  or 
enhancements  were  introduced  in  the  future.  Specifically,  you  are  asked  to  indicate  what  positive  impact,  if  any,  a  given 
item  will  have  on  your  ability  to  carry  out  your  job  more  effectively.  Please  use  the  scale  indicated  here: 


34.  What  positive  impact  would  the  following  features  have 
on  your  ability  to  perform  your  job  more  effectively? 


Major        High         Soma 
Impact      Impact       Impact 


Feature* 

already  Not 

Uttle      available  relevant         Not 

Impact       tome  to  my  area      cure 


a    More  access  to  APS  and  the  walk-up  printers  on 
Saturdays  and  Sundays 


b.    More  access  to  APS  and  the  walk-up  printers  on 
weekday  evenings 


c.    Access  to  patent  material  prior  to  1 971  (text) 


d.    Access  to  electronic  image  search  and  retrieval  (on 
image  workstation) 


e.  More  machines  available  at  peak  usage  periods  in  my 
work  area  (either  shared,  single  screen  text  terminals 
or  image  workstations) 


f.    Connection  of  my  PC  to  the  PTO  network  by  modem 
or  PTO  Net  (text  only) 


g.    Fewer  system  crashes  and  lock-ups,  fewer  times 
kicked  off  the  network 


h.    Faster  log-in  on  the  image  workstation 


i.     Faster  response  time  at  an  image  workstation  (text 
and  image) 


j.     Faster  response  time  at  the  shared,  single  screen 
text  terminals  (text  only) 


k.    Faster  response  time  at  my  desktop  PC  (text  only) 


An  APS  search  routine  that  will  initiate  my  search 
inquiry  within  my  'narrowed'  specification  (class  and 
sub-class),  rather  than  searching  through  all  patents 
all  over  again 


m.  Ability  to  create  and  store  a  personal  digest  of  relevant 
documents  (e.g.,  journal  articles,  foreign  patents,  etc.) 
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FUTURE  NEEDS  ASSESSMENT  CONTINUED 


Features 

already 
available 

Not 

Major 

High 

Some 

Little 

relevant 

Not 

Impact 

Impact 

Impact 

Impact 

tome 

to  my  area 

ture 

n.    A  thesaurus/registry  feature  that  would  automatically 
pull  up  all  similar  terms  and  spellings  within  any  art 
area 


o.    Ability  to  search  a  single  European  patent  database 
electronically 


p.    Ability  to  search  Canadian  patents  electronically 


q.    Ability  to  access  foreign  art  on  the  image  workstation 


r.    An  electronic  lexicon,  to  cross-reference  foreign  terms 
and  their  English  equivalents 


s.    Ability  to  cross-reference  foreign  and  U.S. 
classification  systems 


t     Ability  to  search  abstracts  of  foreign  patents 

(European,  Japanese,  Canadian)  on  my  desktop  PC 


u.    Image  and  text  on  my  desktop  PC,  with  the  ability  to 
easily  switch  back  and  forth 


v.    Ability  to  access  all  commercial  databases  on  my 
desktop  PC 


w.  Automatic  log-in,  where  I  can  access  a  variety  of  data- 
bases in  one  session  without  having  to  log  on  and  off 


x.    More  consistent  commands  across  all  databases 
used  at  PTO 


y.    Simpler,  more  user-friendly  commands  tailored  to  the 
way  search  is  commonly  conducted  in  my  group 


z.    More  mouse-driven  convenience  functions  (as  in 
WordPerfect) 


aa  More  functional  integration  between  the  text  screen  and  the 
image  screen  (e.g.,  highlight  on  a  portion  of  text  and  call  up 
associated  images,  or  mouse  to  a  figure  in  image  and  call 
up  associated  text) 


bb.  Ability  to  scroll  up  and  down  on  the  image  workstation, 
rather  than  just  zoom  in  and  out 
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FUTURE  NEEDS  ASSESSMENT  CONTINUED 


Feature* 

already  Not 

Major        High         Some         Utile)      available  relevant        Not 

Impact      Impact      knpact       Impact      tome  tomyarea      euro 


cc.  Ability  to  access  interference  prints  on  the  image 
workstation 


dd.  Ability  to  have  the  search  terms  highlighted  on  the 
print-out  copy  of  the  patent 


ee.  A  more  convenient  (or  automatic)  back-up-and-save 
feature 


ff.    Page  buffering  at  the  shared,  single  screen  text 
terminals  and  image  workstations,  as  well  as  on  my  PC 


gg.  Ability  to  search  on  the  oldest  effective  priority  date 


hh.  Ability  to  search  on  a  mathematical  equation 


ii.    Ability  to  search  on  a  chemical  structure 


jj.    Ability  to  search  on  a  sequence  of  layered  materials 


kk.  Alternative  search  modalities,  such  as  target  search 
(like  Dialog);  freestyle  search  (like  Lexis);  or  fuzzy  logic 


II.    Ability  to  search  two  or  more  words  in  a  sentence,  in 
any  specified  order 


mm.  Ability  to  conduct  large-scale  searches  without 
exceeding  system  capacity 


nn.  Indication  of  the  number  of  pages  that  will  be  printed 
out  before  I  hit  "print'  key 


oo.  A  "break"  key  that  promptly  stops  the  processing 
when  I  hit  it 


pp.  A  "help"  function  key  that  suggests  possible  solutions  to 
the  problem,  then  returns  me  to  the  prompt 


qq.  A  help  line  that  delivers  welNnformed  and  timely  help 
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FUTURE  NEEDS  ASSESSMENT  CONTINUED 


Features 

already  Not 

Major        High         Soma         Utde      available  relevant         Not 

Impact      Impact       Impact       Impact       tome  to  my  area      aura 


rr.   Better  maintenance  of  hardware  in  the  shared  use/ 
cluster  workstations  (CRTs,  terminals,  printers) 

•                                  u 

ss.  Better  maintenance  at  the  walk-up  pnnters  (paper, 

toner,  cieanng  oi  jamsj 

tL    Advanced/refresher  training  on  new  APS  features, 
tailored  to  the  way  search  is  conducted  in  my  area 

uu.  An  on-line  tutorial,  to  acquaint  me  with  the  latest 
features 

w.  More  practice  time  to  learn  APS  features 

ww.  More  opportunity  to  swap  ideas  among  examiners  on 
how  to  conduct  search 

xx.  A  more  deafly-defined  "pipeline"  for  communicating  -  and 
getting  feedback  on  -  suggestions  for  improving  APS 

yy.  The  ability  to  place  notes  on  electronic  files 

zz.  Capability  to  save  the  NDC  anytime  in  the  search 
session 

aaa  An  assignee  search  function  that  really  works 

bbb.  Notation  of  where  hits  come  from,  in  the  case  of 
merged  sub-classes 

ccc.  E-mail,  to  facilitate  internal  communication  and 
transfer  of  documents 
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35.  Now  that  you've  had  a  chance  to  work  through  the  latter  list  of  possible  enhancements,  please  go  back  over  that  list  one 
more  time  and  indicate  here  which  items  in  your  opinion  would  have  the  most  impact  on  your  job  performance.  (Write  in 
the  corresponding  alphabetical  symbol.) 

a    Most  positive  impact  item: 


b.  Second  most  positive  impact  item: 

c.  Third  most  positive  impact  Item:  _ 


Commercial  Databases 


36.  Do  you  cunently  have  access  to  one  or  more  commercial  databases? 


Yes  (skip  to  quest  on  39) 


No  (answer  questions  37  and  38,  then  skip  to  question  49) 


37.  Based  on  what  you  know  now,  would  you  want  access  to  one  or  more  commercial  databases,  if  you  could  have  it? 


Definitely  yes 


Probably  yes 


Not  sure 


Probably  no 


Definitely  no 


38.  If  you  were  given  access  to  one  or  more  commercial  databases,  what  do  you  think  would  be  your  overall  usage  of  those 
databases? 


Very  heavy  (nearty 
everyday) 


Heavy  (one  or  more 
times  per  week) 


Medium  (1-3  times 
per  month) 


Light  (less  than 
once  per  month) 


None 


(Now  skip  to  question  49) 
39.  From  where  do  you  currently  have  access  to  the  following  commercial  database(s)? 


At  a  special  center  At  the  Scientific  and  Technical 

near  my  work  area  Information  Center 


At  my  desktop  PC 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 
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40.  What  has  been  your  level  of  usage  of  commercial  database(s)  during  the  past  three  months? 


Very  heavy  (nearly  Heavy  (one  a  more    Medium  (1 -"3  limes        Light  (less  ttian 

everyday)  times  per  week)  per  month)  once  per  month) 


None 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 


41 .  How  satisfied  are  you  with  the  availability  of  your  commercial  database(s)  when  you  need  to  use  it  (them)? 


Very 
satisfied 


Generally 
satisfied 


Generally 
dissatisfied 


Very 
dissatisfied 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 

/  satisfied  are 

you 

with  the 

log 

on  /connect  time  required  to  initiate  a  search  of  the  commercial 

databas 

Very 
satisfied 

Generally 
satisfied 

Generally                   Very 
dissatisfied               dissatisfied 

CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 
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43.  How  satisfied  are  you  with  the  system  response  time  required  to  execute  a  search  and  go  to  the  first  screen  while  doing 
a  search  of  your  commercial  database(s)? 

Very  Generally  Generally  Very 

satisfied  satisfied  dssafisfed  dssalisSed 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 

44.  How  satisfied  are  you  with  system  response  time  required  to  go  screen-to-screen  while  doing  a  search  of  your  commer- 
cial database(s)? 

Very  Generally  Generally  Very 

safsfed  satisfied  dssafisfed  dssafisfed 

CAS/STM 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 

45.  How  satisfied  are  you  with  the  overall  user-friendliness  (ease  of  operation)  involved  in  using  your  commercial 
database(s)? 


Very  Generally  Generally  Very 

safsfed  safsfed  dssafisfed  dssafisfed 


CAS/SIN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 
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46.  How  satisfied  are  you  with  the  initial  training  you  received  on  how  to  use  your  commercial  database(s)? 


Very 
safsfed 


Generally 
satisfied 


Generally 
(fssatisfed 


Very 
rfssatisfed 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 


47.  How  satisfied  are  you  with  the  amount  and  quality  of  documentation  you  have  on  how  to  use  your  commercial 
database(s)? 


Very 
safsfed 


Generally 
safsfed 


Generally 
(fssatisfed 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 


48.  To  what  extent  does  your  commercial  database  fulfil  the  information  needs  of  your  job? 


Very 
(fssatisfed 


CAS/STN 

Derwent 

Dialog 

Lexis 

Westlaw 

Orbit 

Other 


Completely 


To  a  large  extent        To  some  extent 


Verylrffle 


Not  at  ail 
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DEMOGRAPHICS 

The  following  information  is  needed  to  help  us  with  the  statistical  analyses  of  all  questionnaires.  All  of  your  responses  are 
confidential  and  cannot  be  associated  with  you  individually.  Your  responses  will  not  be  seen  by  anyone  within  PTO. 

49.  Which  examining  group  are  you  currently  assigned  to? 


1100 
1200 
1300 
1500 
1800 
2100 
2200 
2300 
2400 


2500 
2600 
2900 
3100 
3200 
3300 
3400 
3500 


50.  What  is  the  four  digit  number  of  the  art  unit,  i.e.,  the  examining  subgroup,  you  are  currently  assigned  to? 


51 .  Do  you  consider  your  art  to  be  predominantly . . 


Chemical 


Bectrica! 


Mechanical 


52.  How  would  you  describe  your  current  computer  expertise? 


Complete  beginner 
Novice  -  still  much  to  learn 
Intermediate  user  -  fairly  knowledgeable 
Advanced  user  -  very  knowledgeable 
Computer  pro  -  extremely  knowledgeable 


53.  How  long  have  you  worked  with  the  PTO  as  a  patent  examiner? 


Under  1  year 


1  to  3  years 


3to5years 


5  to  10  years 


Over  10  years 


54.  Are  you  a  primary  examiner? 


Yes 


No 


55.  Are  you  a  supervisory  patent  examiner? 


Yes 


No 
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DEMOGRAPHICS  CONTINUED 


56.  How  long  have  you  been  a  federal  employee? 


Under  1  year 


57.  What  is  your  pay  grade? 


1  to  3  years 


3to5years 


Grades  5-8  Grades  9-12 


Grade  15  and 
Grades  13-15         over(orSES) 


5  to  10  years 


Ungraded 


Ovw  10  years 


Other 


58.  Do  you  usually  work?  (Check  all  that  apply.) 


Regular  business 
hours 


Early  weekday 

mornings 


Weekday 
evenings 


Saturdays 


Sundays 


59.  What  level  of  education  have  you  completed?  (Check  all  that  apply.) 

Graduated  from  college  (B  A,  B.S.,  or  other  Bachelor's  degree) 
Some  graduate  school 
||§  Law  school  graduate  (LLB) 

Graduate  or  other  professional  degree  (MA,  MS,  MBA,  Ph.D,  etc.) 

60.  What  is  your  age? 

20-29  3039  40-49  50-59  60^4 


65  or  older 


A-109 


23 


COMMENTS 

Please  do  not  hesitate  to  elaborate  on  your  responses  to  any  of  the  questions  in  this  census. 


Reference 
Page 

Question 
No. 

THANK  YOU 

We  appreciate  you  taking  the  time  to  share  your  observations  and  opinions  with  us.  When  you  are  finished,  please  place 
the  questionnaire  in  the  enclosed  envelope,  seal  it  and  mail  it  to  us  promptly  at  the  pre-addressed  location.  No  postage  is 
required.  A-110 
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UNITED  STATES  DEPARTMENT  OF  COMMERCE 
Patent  and  Trademark  Office 

ASSISTANT  SECRETARY  AND  COMMISSIONER 
OF  PATENTS  AND  TRADEMARKS 
Washington,  D.C.  20231 


MEMORANDUM 


TO: 


Patent  Examiners 


FROM:  Assistant  Secretary  of  Commerce  and 

Commissioner  of  Patents  and  Trademarks 

The  enclosed  questionnaire  is  part  of  our  commitment  to  solicit  your 
feedback  on  determining  the  best  tools  and  support  for  you  to  do  your 
job. 

I  am  asking  you  to  spend  a  few  minutes  of  your  time  to  share  your 
candid  observations  about  the  Automated  Patent  System  (APS). 
Results  from  this  study  will  provide  essential  information  on  the  impact 
of  APS  on  your  work  on  a  day-to-day  basis  and  will  provide  guidance  on 
any  changes  we  need  to  make  to  better  meet  your  needs.  I  strongly 
encourage  you  to  express  your  ideas  and  report  your  first-hand 
experiences  through  this  process.  Additionally,  information  generated 
from  this  study  will  contribute  to  the  dialogue  on  how  future  automation 
technology  should  be  developed  for  your  use.    The  results  of  this  study 
will  be  tabulated  by  the  end  of  the  year  and  will  be  available  to  all  of  you. 

All  responses  to  the  questionnaire  are  confidential.  There  are  no  coding 
or  tracking  systems  that  would  allow  the  identification  of  individual 
employees. 

Thank  you  for  your  contribution  to  this  effort. 


uAn^K^ 


Bruce  A.  Lehman 


Enclosure 
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UNITED  STATES  DEPARTMENT  OF  COMMERCE 
Patent  and  Trademark  Office 

ASSISTANT  SECRETARY  AND  COMMISSIONER 
OF  PATENTS  AND  TRADEMARKS 
Washington.  D.C.  20231 


OctoLer  13,  1994 

Dear  PATENT  EXAMINER: 

In  late  October  you  will  be  asked  to  complete  a  detailed  questionnaire  designed  by  an 
independent  organization  engaged  by  tbe  Patent  and  Trademark  Office  (PTO)  to  solicit 
your  views  on  the  Automated  Patent  System  (APS).     The  project  is  being  initiated  by 
Commissioner  Lehman  and  is  part  of  our  commitment  to  provide  tne  best  tools  and 
support  you  need  to  do  your  job. 

Tne  senior  management  of  PTO  believes  that  you  —  tne  primary  users  of  APS  —  have 
much  to  tell  us  about  how  APS  is  currently  working  and  what  kinds  of  things  need  to  be 
changed  in  order  to  facilitate  the  way  you  approach  your  work.    We  are  eager  to  hear 
feedback  directly  from  the  examining  corps  and  I  encourage  every  examiner  to  take  the 
time  to  complete  the  questionnaire. 

All  oi  you  will  receive  a  copy  of  the  questionnaire.    Some  of  you  will  be  randomly 
chosen  to  complete  the  questionnaire  in  a  group  setting  in  the  Patent  Academy  on 
October  27.   As  part  of  our  continuous  quality  improvement  effort,  we  will  also  ask 
those  individuals  to  complete  a  survey  on  their  views  of  the  P4000  computers.    The  rest 
of  you  will  receive  the  APS  questionnaire  in  a  sealed  envelope  mailed  to  your  home.     In 
either  case,  you  will  not  be  asked  to  reveal  any  identifying  information  and  your 
responses  will  be  totally  anonymous.     Those  receiving  the  questionnaire  in  the  mail  will 
be  provided  with  an  envelope  for  returning  the  completed  questionnaire  directly  to  the 
outside  organization  we  have  engaged  for  this  study. 

Results  derived  from  the  questionnaire  will  be  published  in  a  report  in  December  1994 
and  copies  made  available  to  you.  What  we  learn  from  your  participation  will  guide  us 
as  we  plan  together  for  the  future. 

Thank  you  for  your  full  cooperation  in  this  important  project. 

Sincerely, 


Lawrence' J.  Goffney,  Jr.     v 
Assistant  Commisisoner  for  Patents 
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UNITED  STATES  DEPARTMENT  OF  COMMERCE 
Patent  and  Trademark  Office 

ASSISTANT  SECRETARY  AND  COMMISSIONER 
OF  PATENTS  AND  TRADEMARKS 
Washington.  DC.  20231 


October  18,  1994 

TO:  GROUP  DIRECTORS 


FROM: 


Office  of  Assistant  Commissioner  for  Patents 


SUBJECT:   APS  Survey 

As  I  indicated  in  my  memo  of  October  13th,  I  consider  the  APS  survey 
an  important  opportunity  for  examiners  to  voice  their  opinions  on  the 
APS  and  the  future  of  automation  in  the  Patent  and  Trademark  Office. 
I  encourage  all  those  who  will  be  surveyed  to  participate. 

Attached  is  a  listing  of  examiners  in  your  Group  who  have  been 
randomly  selected  to  participate  in  the  APS  Survey  on  Thursday 
October  27,  1994  at  2  PM  in  the  Patent  Academy  (PK1,  Suite  501).  Also, 
attached  is  a  memo  for  distribution  to  each  examiner  on  the  list. 

I  ask  that  you  deliver  this  memorandum  to  each  listed  examiner  in  your 
Group  and  confirm  that  they  can  attend.    Once  you  have  confirmation 
on  who  will  attend,  please  fax  that  list  to  Greg  Mullen  from  the  Center 
for  Quality  Services  at  305-8002.    He  needs  your  list  by  Monday 
October  24th,  1994. 

Greg's  fax  number  is  305-8002  and  his  phone  number  is  305-4217,  if  you 
have  any  questions. 


Larry  J.  Goffney,  Jr. 

Assistant  Commissioner  for  Patents 


Attachments 
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UNITED  STATES  DEPARTMENT  OF  COMMERCE 

Patent  and  Trademark  Office 

ASSISTANT  SECRETARY  AND  COMMISSIONER  OF 

PATENTS  AND  TRADEMARKS 

Washington,  D.C.    20231 


October  18,  1994 


Dear  Patent  Examiner: 

You  have  been  randomly  selected  to  participate  in  the  APS  Survey 
Session  to  be  held  on  October  27th,  1994.   As  I  stated  in  my 
letter  to  you  on  October  13th,  I  consider  your  opinion  important 
in  planning  for  the  future  of  automation  in  the  Patent  and 
Trademark  Office.   I  encourage  you  to  take  advantage  of  this 
opportunity  to  let  your  views  be  known. 

Please  advise  your  SPE/Group  Director  on  whether  or  not  you  will 
be  available  to  participate  in  the  survey. 

If  you  are  available  to  participate,  you  are  requested  to  report 
to  the  Patent  Academy  (PK1,  Suite  501)  on  Thursday,  October  27, 
1994,  at  10  AM,  to  take  the  survey. 


Lawrence  J.  Goffney,  Jr. 
Assistant  Commissioner  for  Patents 
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Business  Reengineering  Team.   Patents  As-ls  Activity  Model:  Reader  Kit.   U.S. 
Patent  and  Trademark  Office,  U.S.  Department  of  Commerce,  April  11,  1994. 

A  working  document  prepared  by  PTO's  Business  Reengineering  Team,  the 
Patents  As-ls  Activity  Model:  Reader  Kit  describes  in  narrative  and  work-flow  form  all 
activities  associated  with  the  patent  application  process.   This  particular  version  of  the 
document,  the  "recommended  version,"  was  distributed  to  select  internal  readers  for 
comment. 

Cohen,  Senator  William  S.   Computer  Chaos:  Billions  Wasted  Buying  Federal 
Computer  Systems.   Subcommittee  on  Oversight  of  Government  Management,  Senate 
Governmental  Affairs  Committee,  October  12,  1994.   (32  pages) 

A  comprehensive  report  on  the  federal  government's  practices  for  purchasing 
computers,  Computer  Chaos  outlines  the  major  problems  associated  with  the  federal 
government's  computer  acquisition  strategies,  and  makes  several  recommendations 
on  how  these  practices  should  be  changed.   The  report  emphasizes  the  need  for 
earlier  and  more  effective  oversight  of  computer  acquisition  programs. 

Lucas,  Jay,  Mathews,  Alan,  Purcell,  Arthur,  Stouffer,  Richard  and  Swann,  Judy. 
Analysis  of  Advantages  of  APS.   Office  of  Planning,  Development  and  Evaluation, 
Search  and  Information  Resources  Administration,  Patent  &  Trademark  Office,  March 
31,  1994.  (29  pages) 

Based  on  comments  by  working  examiners  and  classifiers,  as  well  as 
observations  by  Search  and  Information  Resources  professionals  who  have  been 
engaged  in  the  development  of  many  of  APS's  features,  this  report  outlines  some 
advantages  of  using  APS's  Classified  Search-Image  Retrieval  and  Text  Search 
subsystems  over  the  paper-based  shoes.   The  report  is  an  update  of  a  paper  written 
in  January,  1993  and  is  divided  into  the  following  sections:   Quality  Advantages, 
Patent  Application  and  Patent  Processing  Advantages,  Time  Saving  Advantages, 
Public  Service  and  U.S.  Competitiveness  Advantages,  International  Agreement 
Advantages,  and  other  advantages  of  APS  over  the  paper  shoes. 
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U.S.  Patent  and  Trademark  Office.    Strategic  Information  Technology  Plan:   FY1995- 
FY2000.    U.S.  Department  of  Commerce,  June  24,  1994  and  October  24,  1994  Drafts. 

These  draft  documents  outline  the  five  major  automated  information  systems 
which  make  up  the  Automated  Patent  System  and  describes  the  development, 
replacement  and  enhancement  projects  that  are  planned  for  the  period  FY1995  to 
FY2000. 

U.S.  Government  Accounting  Office,  Report  to  Congressional  Requestors.    Patent  and 
Trademark  Office:  Key  Processes  for  Managing  Automated  Patent  System 
Development  Are  Weak.   September,  1993. 

In  response  to  a  request  by  The  Honorable  Dennis  DeConcini,  Chairman  of  the 
Subcommittee  on  Patents,  Copyrights  and  Trademarks,  Committee  on  the  Judiciary, 
U.S.  Senate,  and  The  Honorable  William  J.  Hughes,  Chairman  of  the  Subcommittee 
on  Intellectual  Property  and  Judicial  Administration,  Committee  on  the  Judiciary,  U.S. 
House  of  Representatives,  the  GAO  prepared  this  report  on  the  PTO's  ability  to 
establish  benefit,  cost  and  schedule  baselines  for  the  Automated  Patent  System.   The 
report  concludes  that  PTO  does  not  have  the  appropriate  processes  in  place  to 
manage  APS  in  a  sound  and  efficient  manner,  and  makes  four  major 
recommendations  for  determining  whether  or  not  PTO's  current  objectives  for  APS 
development  are  relevant  to  what  the  office  needs  in  terms  of  an  automated 
information  system. 
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